Next Article in Journal
How Much Environment Do Humans Need? Evidence from an Integrated Online User Application Linking Natural Resource Use and Subjective Well-Being in Germany
Previous Article in Journal
Aquatic-Derived Biomaterials for a Sustainable Future: A European Opportunity
Previous Article in Special Issue
Proposal of a Sustainable Circular Index for Manufacturing Companies
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Developing Composite Indicators for Agricultural Sustainability Assessment: Effect of Normalization and Aggregation Techniques

1
McGill Centre for the Convergence of Health and Economics (MCCHE), McGill University, Montreal, QC H3A 1X9, Canada
2
Department of Systems Design Engineering, University of Waterloo, Waterloo, ON N2L 4B6, Canada
3
Centre for International Governance Innovation and the Balsillie School of International Affairs, Waterloo, ON N2L 6C2, Canada
4
School of Environmental Studies, Queen’s University, Kingston, ON K7L 3N6, Canada
*
Author to whom correspondence should be addressed.
Resources 2017, 6(4), 66; https://doi.org/10.3390/resources6040066
Received: 26 August 2017 / Revised: 18 November 2017 / Accepted: 19 November 2017 / Published: 24 November 2017
(This article belongs to the Special Issue Sustainability Indicators for Environment Management)

Abstract

:
The assessment of the sustainability of agricultural systems is multidimensional in nature and requires holistic measures using indicators with different measurements and units reflecting social, economic, and environmental aspects. To simplify the assessment process, various indicators have different units, and measurements are grouped under broad indicator heads, and normalization and/or transformation processes are carried out in order to aggregate them. In this study, a total of 50 indicators from agricultural sustainability categories of productivity, stability, efficiency, durability, compatibility, and equity are employed to investigate which normalization technique is the most suitable for further mathematical analysis for developing a final composite indicator. To understand the consistency and quality of normalization measurement techniques and compare the benefits and drawbacks of the various selected normalization processes, the indicators of agricultural sustainability are considered. Each of the different techniques for normalization has advantages and drawbacks. This study shows that the proportionate normalization and hybrid aggregation rules of the arithmetic mean and the geometric mean are appropriate for the selected data set, and that this technique has a wider applicability for developing composite indicators for agricultural sustainability assessment.

1. Introduction

Assessing agricultural sustainability holistically requires multidimensionality [1], and therefore a set of appropriate social, economic, and ecological indicators are needed [2,3]. Multiple criteria methods [4,5] and different kinds of indicators have been designed to evaluate agricultural sustainability [3,6,7]. A multidimensional set of indicators of agricultural sustainability can be interpreted using different statistical scales such as ordinal and nominal, which may be difficult to handle. To avoid this problem, a multidimensional set of indicators can be aggregated into a composite indicator [8]. Andreoli and Tellarini [9], Pirazzoli and Castellini [10], Rigby et al. [11], and van Calker et al. [12] have utilized this approach for agricultural sustainability assessment. A number of steps must be followed to develop a composite indicator. According to the Organization for Economic Co-operation and Development [13], there are 10 steps for constructing ideal composite indicators: theoretical framework, data selection, imputation of missing data, multivariate analysis, normalization, weighting and aggregation, uncertainty and sensitivity analysis, back to the data, links to other indicators, and visualization of the results. These 10 steps are well documented ([13], pp. 20–22, see Table A1 in Appendix A); the normalization step is particularly important for constructing composite indicators.
In the case of agricultural sustainability assessment, the indicators are rarely in the same measurement units irrespective of scholars and organizations [2,3], so normalization for developing composite indicators requires special care. Applying the methodological process of constructing composite indicators in the coastal agricultural systems of Bangladesh as a case study, this paper aims to show the differences between normalization and aggregation techniques and their impacts on the resulting rankings of the composite indicators of the sustainability of agricultural systems. Mathematical experiments are carried out to test whether the ranking of the composite indicators is greatly influenced by the choice of a normalization technique. These tests are important in a situation in which there are no reference values or goalpost values for developing composite indicators to assess the sustainability of agricultural systems. This paper highlights the advantages and disadvantages and compares the results obtained when applying these tests to a dataset containing selected agricultural sustainability indicators for coastal Bangladesh. A point to be noted here is that composite indicators for agricultural sustainability assessment are still the subject of policy/strategic discussion or documents in Bangladesh. There have been a few studies [14,15,16] that measured aspects of the sustainability of agricultural systems in Bangladesh in a very isolated way, but the measures themselves have rarely been compared. Although a few studies in Bangladesh have used indicators for assessing agricultural sustainability, this is the first one in Bangladesh that checks the effects of normalization and aggregation techniques on developing composite indicators.

Brief Overview of Composite Indicators

The concept of composite indicators was introduced in the 1990s to capture the complexity and multidimensionality of a range of development issues [17]. Since then, international organizations like the United Nations, World Bank, and European Commission have developed composite indicators [18] such as the Human Development Index (HDI), Environmental Performance Index (EPI), Gender Empowerment Measure (GEM), and Quality of Life Index. In the literature, the term “composite indicator” often refers to an index made up of aggregated data, ratings, league tables, and multidimensional measures [19,20,21]. Bandura and Martin del Campo (as cited in [18]) found 160 composite indicators used around the world.
Although composite indicators are being used extensively, there is a spirited debate over the conceptual and methodological parameters for this measurement technique [22]. For example, Sharpe [23] argued that producing a composite indicator/index is not a good idea because a single indicator is not appropriate to explain and compare any observed phenomenon and does not capture the relative importance of the components of the composite indicators [20]. In spite of this limitation, composite indicators are considered to be desirable among policy makers and stakeholders due to their capacity to summarize complex issues [24], allow for cross comparisons, enable evaluation of results, set the bar for performance, and indicate the steps of accomplishment of a project [25]. They are also useful for generating media interest about a phenomenon [23]. Comprehensive discussions of the advantages and disadvantages of composite indicators are documented in Booysen [26], Foa and Tanner [18], and Nardo et al. [20].
Conceptually, composite indicators are based on sub-indicators that may have no common meaningful unit of measurement [20]. Technically, composite indicators are mathematical combinations of a set of multidimensional indicators [20,24] and normal measures that combine the issues of a complex phenomenon [26]. Therefore, the construction of composite indicators requires transparency as to its process to facilitate replication and debate among stakeholders [24]. The construction of composite indicators requires more craftsmanship by the modeler than universally accepted scientific rules for encoding indicators [20]. Basically, a typical composite indicator “ I ” is built as follows [13]:
I = i = 1 n w i x i where x i              = normalized variable w i             = weight   attached   to x i i = 1 n w i = 1   and   0 w i 1 , i = 1 , 2 , , n .
From this formula, it is clear that a composite indicator requires a weighted linear aggregation rule that is applied to a set of variables. The formula indicates that normalization and weighted summation of the normalized variables are the two main steps for developing composite indicators.
Data can be aggregated without being scaled if all the variables are measured with the same unit (e.g., percent or ratios), but in many situations the variables to be aggregated have different units and different measurement techniques [27] such as nominal, ordinal, interval, and ratio scales. In this situation, normalization is the process by which the indicators in various scales and units are compared on a common basis, as depicted in Figure 1. Normalization is, therefore, the process of reducing the measurements to a standard scale [28], which helps to avoid the dominance of extreme values in a data set and partially corrects data quality problems [29]. Normalization of indicators is required to make the indicators mathematically operational in aggregation [8].
Every step of data transformation and/or normalization increases the probability of uncertainty and measurement error [30]. Accordingly, the choice of the proper normalization technique is indisputably important. In developing composite indicators, the selection of a preferred normalization technique deserves special care, taking into account the objectives of the composite indicators as well as the data properties and the potential requirement of further analysis [20,31]. Different normalization techniques produce different results [13] and may have major effects on composite scores [22,32].

2. Materials and Methods

For this paper, five normalization techniques are examined to investigate their effect and to identify the preferred technique for constructing composite indicators of coastal agricultural sustainability assessment in Bangladesh. Figure 2 shows the construction and evaluation process of the individual composite indicators that are examined in this paper. As shown in this figure, sustainability was categorized in terms of productivity, stability, efficiency, durability, compatibility, and efficiency. In brief, productivity is related to the yield of agricultural systems, stability refers to the ability to maintain a good level of productivity over an extended period of time, and efficiency is the measure of the extent to which the inputs for agricultural production enhance the crop yield. Durability can be defined as the ability of the agricultural system to resist or recover from stress and thereby maintain a good level of productivity over a cropping cycle. Compatibility refers to the ability of an agricultural system to fit in with the bio-geophysical, human, and socio-cultural surroundings in which the system is placed, and equity reflects a good quality of life for farmers and their family members [7]. For details about these categories, see vanLoon et al. [7]. For conceptual judging and selection of indicators, the dataset of Talukder [33] was used. Talukder [33] developed 110 indicators for assessing the sustainability of the coastal agricultural systems of Bangladesh. From 110 indicators of six sustainability categories, 50 indicators (Table A2, Table A3, Table A4, Table A5, Table A6 and Table A7 in Appendix B) were judged according to their importance and then grouped to make 15 composite indicators, as depicted in Figure 2. Then, various normalization, weighting, and aggregation techniques were applied to identify suitable normalization and aggregation techniques for developing the final set of composite indicators.

2.1. Overview of Datasets

The datasets in Table A2, Table A3, Table A4, Table A5, Table A6 and Table A7 in Appendix B contain different measurement units under six categories of sustainability. The data were collected from both primary and secondary sources in the southwest coastal zone of Bangladesh. The primary data were collected from five different agricultural systems: Bagda (shrimp)-based agricultural systems (S) from Shyamnagar; Bagda-rice-based agricultural systems (SR) from Kalijang; rice-based agricultural systems (R) from Kalaroa; Galda-rice-vegetable-based integrated agricultural systems (I) from Dumuria; and traditional practices-based agricultural systems (T) from Bhola Sadar (Figure 3). The details of the data collection process, justification of data collection, and development of indicators can be found in Talukder [33]. The description of the indicators, their units, data type, their relationships with sustainability pillars, data collection areas, data sources, and levels of measurement are presented in brief in Table A2, Table A3, Table A4, Table A5, Table A6 and Table A7 in Appendix B. A point to be noted here is that to develop the indicators, the collected data were processed through cleaning, integration, reduction, and transformation. No outliers were detected in the collected data during these processes.

2.2. Normalization

A variety of transformation and/or normalization techniques are available (e.g., [29,34,35,36]), but only the five most widely employed techniques [35,37] are shown in Table 1. These five techniques are ranking, distance to target, Z-score, min-max, and proportionate normalization. The first four are the most commonly used normalization techniques [13,21]. The proportionate normalization technique was considered because of its suitability for the development of composite indicators.

2.2.1. Ranking Normalization

Ranking normalization replaces measurements with their rank. In the rank normalization process, each data point is replaced by its rank, that is, by values ranging from 1 (lowest) to N (highest) [38]. In this system, there is no score, only a rank; the absolute-level information is lost. This technique, while simple, cannot lead to any conclusion about the differences among performances of the indicator being assessed because there is no measure of the distance between values of the indicators [39]. Ranking normalization is employed in the “Information and Communications Technology Index” [29] and “Medicare Study on Healthcare Performance across the United States” [40].

2.2.2. Distance to Target Normalization

In the distance to target normalization technique, the indicator’s value is divided by the target value to normalize the indicator [21] so that the normalized values represent a fraction of the highest value. The highest value of the indicator set or any reference point can be the target value. The results of this technique are easy to handle and understand, but imbalance between scores and rankings remains, and the normalization results are more influenced by outliers than in other techniques. This method is useful for further analysis (e.g., geometric aggregation) since it does not generate any zero values. However, if outliers are chosen as target points, the result can be misleading. The distance to target normalization technique is used in “Eco-indicator 99” and the “Summary Innovation Index” [41].

2.2.3. Z-Score Normalization

Z-score normalization is calculated by subtracting the mean from an indicator value and then dividing by its standard deviation. If the standard deviation is calculated for a set of variables with a mean of 0 and then all values are divided by the standard deviation, the resulting set of values will have a standard deviation of 1 [27]. After performing normalization, the data have a common scale with a 0 mean and a standard deviation of 1. Since all Z-score distributions have the same mean and standard deviation, individual scores from different distributions can be directly compared. The advantage of this technique is that it provides no distortion from the mean, adjusting for different scales and variance. The output is dimensionless, and the relative differences are maintained due to the application of a linear transformation [42]. Z-score is preferred when extreme values exist in the dataset [20,32]. Although the technique does not fully adjust for outliers, the minimum and maximum values are not as influential as in other techniques such as distance to target. When extreme values are present in the original data, Z-score normalization takes these extreme values into account in a manner that does not distort their impacts on a composite indicator. In this way, an outlier, such as exceptional performance, is recognized and not ignored [13,27]. The Z-score technique is widely employed, including in the knowledge-based economy index [43] and the World Health Organization’s child growth standards index [44].

2.2.4. Min-Max Normalization

The min-max technique rescales data into different intervals based on minimum and maximum values. The advantage of this method is that boundaries can be set and all indicators have an identical range (0, 1). However, the normalized values do not maintain proportionality, and normalized values reflect the percentage of the range of max as ( X ias ) min as ( X ias ) . This technique is based on extreme values (minimum and maximum), but because these two values can be outliers, the range of max and min strongly influences the final output. Another disadvantage is that the difference in variance is not fully eliminated [13]. Nevertheless, this technique is very popular and has been applied in the construction of many composite indicators, the best-known of which is the Human Development Index (HDI, [45]).

2.2.5. Proportionate Normalization

In proportionate normalization, the single attribute value is divided by the sum total of the values of attributes [37,46]. The normalized values maintain proportionality such that they reflect the percentage of the sum of the total value of the indicators. Here, values of the indicator are relatively normalized. Normalizing the indicators by dividing them by their sums has a number of attractive properties, including that the normalized values are identical to the original, except for a scaling factor, and the process is easily understandable. The value differences among indicators become narrow. Dividing by the sum ensures that even the smallest value greater than zero comes out with a positive normalized value [19,37]. The proportionate normalization technique is frequently used in normalizing census data in ArcView GIS (Geographical Information System, [46]). Benini [19] also suggested using this technique for developing composite measures for disaster impact assessment.

2.3. Weighting

The final score and ranking of the composite indicators depends on the weighting of the normalized values of the indicators. Weighting reflects the importance of each indicator relative to the overall composite indicators [21]. Weights should ideally be selected according to an underlying and agreed-upon, or at least clearly stated theoretical, framework so that the process is transparent [47,48]. Weighting can be a very important step in creating composite indicators before aggregation can take place, because it modifies the sub-indicator values. However, Sajeva et al. [28] have shown that the use of different weighting schemes can often have no significant effect on the ranking of the composite indicators. No agreed-upon methodology exists to weight individual indicators. Different types of weighting techniques and their explanations are provided by Nardo et al. [20].
In this paper, equal weighting of sub-indicators is used for all rank, distance to target, Z-score, max-min and proportionate normalization, and arithmetic mean and geometric mean aggregation. Simplicity is the main advantage of equal weighting, but the composite indicator that is developed by the combination of more indicators will have a stronger influence on the list of composite indicators. Using this weighting system may be justified when no other available means of weighting are known [30]. Equal weighting is used in the HDI [49]. Budget allocation techniques for weighting are used for MCA aggregation (as shown in Table A13). A budget allocation technique for weighting is chosen because the sustainability of agriculture is very contextual, so stakeholders’ opinions are very important for weighting of the indicators. Geometric and multi-criteria, as well as linear, aggregation can be employed with these weightings [13]. The OECD’s Handbook on Constructing Composite Indices [13] describes expert weighting as a budget allocation technique. In expert weighting, an expert allocates 100 points among indicators according to their importance [21]. Selection of the appropriate expert and number of experts is the biggest problem for this system because point allocation may be influenced by the expert’s experience [30]. This subjective judgment of the weights of sub-indicators is used to allocate relative worth for each sub-indicator [22]. Subjective weighting is often affected by strong inter-individual disagreement [29] and is particularly sensitive in the case of complex, interrelated, and multidimensional phenomena [20]. Nevertheless, Sen and Foster ([48], p. 206) pointed out that “while the possibility of arriving at a unique set of weights is rather unlikely, that uniqueness is not really necessary to make acceptable judgments in many situations, and may indeed not even be required for a complete ordering”.

2.4. Aggregation

The rules for aggregation are well documented in the Handbook on Constructing Composite Indices [13], but steps are still debated in the development of composite indicators [50]. The fundamental issue in aggregation is the compensability of indicators, which is defined as compensating for any indicator’s dimension with a suitable surplus in another indicator’s dimension. The rules for aggregating composite indicators can be compensatory or non-compensatory [51]. A compensatory technique deals with the imbalances in the indicators and uses linear functions, whereas non-compensatory techniques use unbalance-adjusted functions [52]. Different aggregation rules are possible to develop composite indicators. Commonly applied aggregation options include additive aggregation (arithmetic mean), geometric aggregation (multiplication), and multi-criteria analysis [13].
The arithmetic mean is a linear function [53]. The normalized and weighted or unweighted indicators are summed to compute the arithmetic mean (the formula for evaluating arithmetic mean is x ¯ = i = 1 n x ¯ N ) [26,32]. In this method, compensability can be a disadvantage if a low value in one indicator or dimension masks a high value in another, that is, a deficit in one indicator or dimension can be compensated for by a surplus in another [30,32].
Geometric aggregation, which is the product of normalized weighted indicators, is used to avoid concerns related to interaction and compensability [32]. Non-comparable data measured in a ratio scale can only be meaningfully aggregated by using geometric functions, provided that indicators are strictly positive [20,30]. A geometric mean (the formula for evaluating geometric mean is ( i = 1 n x 1 ) 1 n ) takes into consideration differences in achievement across dimensions [20]. Poor performance in any dimension or indicator is directly reflected in the composite indicator’s value. According to Hudrlikova and Kramulova [30], this technique is partly compensable since it rewards composite indicators with higher indicator scores.
“When different goals are equally legitimate and important, and in addition trade-offs exist between the dimensions of a composite indicator (namely negative correlations between dimensions) then a non-compensatory logic may be necessary” ([21], p. 256). Multi-Criteria Analysis (MCA) is used for aggregating non-compensatory data [52]. In general, MCA provides an overall ranking based on the weight and values of given indicators. One of the shortcomings of MCA is that when the number of indicators to develop composite indicators is high, it is difficult to compute MCA [30]. MCA is based on an outranking matrix. The standard procedure for performing an MCA consists of three steps: identifying the weighting of the criteria, preparing an “outranking matrix” by pairwise comparison of the weighted performance of each criterion (for n options, there are n (n − 1)/2 comparisons) [54], and calculating the composite indicator score of the criteria by adding the values of the row of the outranking matrix [55].

2.5. Robustness

The outcome of the composite indicators depends on the selection of variables, normalization, weighting (if it is used), and aggregation techniques [20], so it is necessary to examine the robustness of the developed composite indicators. Various statistical tests can help ensure that the composite is reliable. Freudenberg [29] and Hudrlikova and Kramulova [30] mentioned correlation as a technique to assess the impacts of different normalization techniques on composite indicators. The correlation coefficient can show whether the results of the composite indicator are heavily influenced by the choice of normalization rules [30] and aggregation methods. In this paper, correlation is used to assess the robustness of composite indicators.

3. Results

The results for the composite indicators using various normalization techniques, weighting, and different aggregation techniques are presented in Table A8, Table A9, Table A10, Table A11, Table A12 and Table A13 in Appendix B. The results of the robustness tests of the composite indicators are presented in Table A14, Table A15, Table A16, Table A17, Table A18, Table A19, Table A20, Table A21, Table A22, Table A23, Table A24, Table A25, Table A26 and Table A27 in Appendix B.
The values of data and different normalization techniques and arithmetic aggregation involve different assumptions that have specific consequences that produce different results for the composite indicators (Table A8, Table A9, Table A10, Table A11, Table A12 and Table A13). In this regard, Nardo et al. [20] mentioned that the ranking of composite indicators is heavily influenced by the nature of the data. Saisana and Saltelli [21] also pointed out that it is beyond doubt that composite indicators are a value-laden construct. In arithmetic aggregation, it is also observed that poor performance in some indicators is covered by sufficiently high values of other indicators in composite indicators.
In the dataset for this study, the score for some of the indicators is “0”. For example, as shown in Table A6, in the compatibility category “S” scored “0” in drinking water quality. Indicators that have “0” scores have the normalization result “0” in proportionate, distance, and Z-score normalization, but not a “0” ranking normalization, since the score “0” is ranked as the lowest number. The max-min normalization also generates “0” scores as normalized values. Whenever the normalization score is “0” or negative, those indicators are not suitable for geometric mean aggregation because geometric aggregation requires all positive numbers and is therefore only appropriate when indicator values are always positive [20].
When aggregation was carried out considering indicators’ values and budget allocation weight and MCA techniques, the results also generated different values for some of the composite indicators (Table A13) compared to other types of aggregation. Due to the nature of the data, MCA also generates “0” values for productivity, energy efficiency, and human compatibility composite indicators of “S”, as well as “0” values for resistance to economic stress, resistance to climate change, and gender composite indicators of “T” (Table A13). Therefore, budget allocation weighting and MCA combinations cannot be recommended for composite indicators.
These different values of the composite indicators that result from applying different combinations of normalization techniques, weighting, and aggregation reflect that the properties of the indicators are very crucial for the final output values of the composite indicators. This study shows that the normalization technique, arithmetic mean, and geometric mean should take into account the data properties, as well as the objectives of the composite indicator. From the results, it appears that not all normalization techniques are suitable for the dataset, and not all normalization techniques support arithmetic mean and geometric mean. Even when MCA techniques are applied, some “0” values are generated for the composite indicators.
Nardo et al. [20] suggested that in the case of non-compensatory composite indicators, MCA is the best way to develop indicator values. However, due to the nature of the present dataset, MCA is not suitable for this experiment because the “0” scores of some of the indicators do not reflect the weight of the indicator, so the results may be difficult to interpret and compare. In MCA, composite indicators are based on weight, so the magnitudes of values of the different indicators are disregarded in the composite. “This means any issue that does marginally better on many indicators score higher than the issue that does a lot better on a few indicators because outstanding performances of the indicators cannot compensate for the deficiencies in some indicators” ([56], p. 364).

4. Discussion

In this study, it is observed that proportionate normalization produced values that conserve the proportionality of the indicator values (Table A12), whereas other normalization techniques show different outcomes. For example, the normalization results of rank, distance to target, Z-score, max-min, and proportionate normalization for weighted yield of rice indicators of “S” in Table A8, Table A9, Table A10, Table A11 and Table A12 are 1, 0.35, −1, 0 and 0.11, respectively. Here, only the 0.11 that is generated using proportionate normalization represents the proportionate value of the original score of 2.26 for the weighted yield of rice indicator of “S” (see Table A2). Therefore, proportionate normalization is selected to develop composite indicators in this study, because the original values of the data do not change through this process. If the values of the data change due to the transformation technique/normalization, they are mathematically not meaningful. Therefore, it is always preferable to follow a technique by which original data are transformed in such a way that their informational content is not fundamentally altered [22]. In proportionate normalization, the rank of the composite indicator depends on actual values since proportionate normalization does not alter the actual importance of the values of the indicators. This is the strength of this technique [22]. Furthermore, proportionate normalization seems preferable in this experiment to the most popular min-max normalization because there are no goalpost values for any of the 50 indicators.
There is clearly no universal best aggregation method because aggregation depends on the requirement of the developer of the composite indicators. In the data there are some “0” values. Therefore, to aggregate the indicators, a hybrid aggregation is suggested: indicator values with the “0” normalization result will be aggregated by arithmetic mean, and the rest will be aggregated by geometric mean. Hybrid aggregation techniques use more than one aggregation function at different levels [54]. For example, the “Multidimensional Poverty Assessment Tool (UNIFAD, 2010 as cited in [54]) used arithmetic average within a subcomponent and geometric average within a component, while the Food and Nutrition Security Index (FAO, 2014 as cited in [57]) used arithmetic averages within dimensions and geometric average across dimensions” ([57], p. 16). When comparing all applied normalization and aggregation techniques, it appeared that, for the present research, the proportionate normalization and hybrid aggregation techniques (geometric mean and arithmetic mean) produced the most preferred results. Therefore, 15 single composite indicators are developed from the 50 indicators in Talukder et al. [16] using proportionate normalization and hybrid aggregation. These 15 single composite indicators (see Table 2) are proposed to create a set of the most representative variables of agricultural sustainability in the study area. Among these 15 composite indicators, “monetary efficiency” carries the proportionate normalization values of the original values without any aggregation but is normalized by proportionate normalization.

5. Conclusions

This study tested various normalization and aggregation techniques for developing composite indicators, providing a comparison among different combinations to find out the best normalization and aggregation combination. Normalization techniques, weighting, and aggregation all influence the final outcomes of composite indicators, so it is important to compare different combinations of normalization, weighting, and aggregation techniques. Rank, distance to target, Z-score, max-min, and proportionate methods were used for normalization, while equal weight and budget allocation for weighting and arithmetic mean, geometric mean, and multi-criteria analysis were used for aggregation. The results show that the normalization and characteristics of data have a huge influence on composite indicators. For example, the human compatibility composite indicator in the compatibility category has a score of “0” using rank normalization and geometric aggregation, distance to target normalization and geometric mean, Z-score normalization and arithmetic mean, Z-score normalization and geometric mean, max-min normalization and arithmetic mean, proportionate normalization and geometric mean, proportionate normalization and arithmetic mean, or MCA. A score of 1 results from using rank normalization and arithmetic mean, a score of 0.25 from using distance to target normalization and arithmetic mean, and a score of −1.98 using Z-score normalization and arithmetic mean.
Both methodological and empirical conclusions can be drawn from this study. From a methodological point of view, it can be said that proportionate normalization and the hybrid aggregation technique are suitable for developing composite indicators from these empirical data, which are developed through a questionnaire and secondary data and have a score of “0” for several indicators. These techniques allow the aggregation of a multidimensional set of indicators into a unique composite indicator that can facilitate the understanding of a complex concept such as agricultural sustainability. In the case of proportionate normalization, weighting the indicators has no effect. However, these techniques depend on the properties of the indicators, and some subjectivity is associated with the selection of normalization and aggregation rules. Depending on the methodology selected for constructing indicators, the results of the composite indicators can vary and sometimes be misleading. Based on the properties of the dataset, it appears that proportionate normalization is appropriate, and a hybrid of aggregation rules is suitable for developing composite indicators. However, it is the responsibility of the designer of the composite indicator to choose the most appropriate normalization and aggregation techniques. These techniques must have a sound and transparent methodological framework. In this respect Nardo et al. [20] also stated that the selection of the normalization process deserves special care.

Acknowledgments

This paper has been supported by the Social Sciences and Humanities Research Council (SSHRC), Canada.

Author Contributions

Byomkesh Talukder conceived and designed the original research as part of his Ph.D. thesis. Keith W. Hipel helped to execute the research and Gary W. vanLoon guided the statistical techniques to develop criteria.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1. Checklist for building a composite indicator.
Table A1. Checklist for building a composite indicator.
StepsDescriptionWhy It Is Needed
1st: Theoretical frameworkProvides the basis for the selection and combination of variables into a meaningful composite indicator under a fitness-for-purpose principle (involvement of experts and stakeholders is envisioned in this step).
To get a clear understanding and definition of the multidimensional phenomenon to be measured.
To structure the various sub-groups of the phenomenon (if needed).
To compile a list of selection criteria for the underlying variables, e.g., input, output, and process.
2nd: Data selectionShould be based on the analytical soundness, measurability, country coverage, and relevance of the indicators to the phenomenon being measured and their relationship to each other. The use of proxy variables should be considered when data are scarce (involvement of experts and stakeholders is envisioned in this step).
To check the quality of the available indicators.
To discuss the strengths and weaknesses of each selected indicator.
To create a summary table of data characteristics, e.g., availability (across country, time), source, type (hard, soft or input, output, and process).
3rd: Imputation of missing dataIs needed in order to provide a complete dataset (e.g., by means of single or multiple imputation).
To estimate missing values.
To provide a measure of the reliability of each imputed value so as to assess the impact of the imputation on the composite indicator results.
To discuss the presence of outliers in the dataset.
4th: Multivariate analysisShould be used to study the overall structure of the dataset, assess its suitability, and guide subsequent methodological choices (e.g., weighting, aggregation).
To check the underlying structure of the data along the two main dimensions, namely individual indicators and countries (by means of suitable multivariate methods, e.g., principal components analysis, cluster analysis).
To identify groups of indicators or groups of countries that are statistically “similar” and provide an interpretation of the results.
To compare the statistically determined structure of the dataset to the theoretical framework and discuss possible differences.
5th: NormalizationShould be carried out to render the variables comparable.
To select suitable normalization procedure(s) that respects both the theoretical framework and the data properties.
To discuss the presence of outliers in the dataset as they may become unintended benchmarks.
To make scale adjustments, if necessary.
To transform highly skewed indicators, if necessary.
6th: Weighting and aggregationShould be done along the lines of the underlying theoretical framework.
To select appropriate weighting and aggregation procedure(s) that respects both the theoretical framework and the data properties.
To discuss whether correlation issues among indicators should be accounted for.
To discuss whether compensability among indicators should be allowed.
7th: Uncertainty and sensitivity analysisShould be undertaken to assess the robustness of the composite indicator in terms of the mechanism for including or excluding an indicator, the normalization scheme, the imputation of missing data, the choice of weights, the aggregation method, and so forth.
To consider a multi-modelling approach to build the composite indicator and alternative conceptual scenarios for the selection of the underlying indicators if available.
To identify all possible sources of uncertainty in the development of the composite indicator and accompany the composite scores and ranks with uncertainty bounds.
To conduct sensitivity analysis of the inference (assumptions) and determine what sources of uncertainty are more influential in the scores and/or ranks.
8th: Back to the dataIs needed to reveal the main drivers of overall good or bad performance. Transparency is primordial to good analysis and policymaking.
To profile country performance at the indicator level so as to reveal what is driving the composite indicator results.
To check for correlation and causality (if possible).
To identify whether the composite indicator results are overly dominated by a few indicators and to explain the relative importance of the sub-components of the composite indicator.
Source: [13], pp. 20–22.

Appendix B

Table A2. Selected indicators and values to construct single composite indicators for productivity.
Table A2. Selected indicators and values to construct single composite indicators for productivity.
Sustainability CategoryComposite IndicatorDescriptionUnitData TypeSustainability PillarData SourceWeightingStudy AreasLevel of Measurement
SSRRIT
ProductivityProductivityWeighted yield of the main staple cropt/haQTLEconomicQS0.42.264.415.236.512.86Ratio scale
Net income from the agro-ecosystem$/haQTLEconomicQS0.4311.151020.371585.811806.04544.01Ratio scale
Protein yield from the agro-ecosystemkg/haQTLEcologicalQS0.268.42147.23552373.01318.87Ratio scale
Legend: QNT = Quantitative; QS = Questionnaire survey.
Table A3. Selected indicators and values to construct single composite indicators for stability.
Table A3. Selected indicators and values to construct single composite indicators for stability.
Sustainability CategoryComposite IndicatorDescriptionUnitData TypeSustainability PillarData SourceWeightingStudy AreasLevel of Measurement
SSRRIT
StabilityLandscape stabilityLand exposure to natural events: cyclonebinary yes/no responseQUALEcologicalSD0.312221Nominal scale
Land exposure to natural events: saline waterbinary yes/no responseQUALEcologicalSD0.311323Nominal scale
Land exposure to natural events: drought in kharif to rabi seasonbinary yes/no responseQUALEcologicalSD0.051.51.5223.5Nominal scale
Land exposure to natural events: river bank erosionbinary yes/no responseQUALEcologicalSD0.0522221Nominal scale
Stability of embankmentbinary yes/no responseQUALEcologicalFO0.212122Nominal scale
Withdrawal of upstream waterbinary yes/no responseQUALEcologicalSD0.111112Nominal scale
Soil health/stabilityOrganic materials%QTLEcologicalSD0.344232Ordinal scale
SalinitydS/mQTLEcologicalSD0.3515636Ordinal scale
Macronutrient: Nmeq/100 gQTLEcologicalSD0.122212Ordinal scale
Macronutrient: Pmeq/100 gQTLEcologicalSD0.132333Ordinal scale
Macronutrient: Kmeq/100 gQTLEcologicalSD0.164324Ordinal scale
Soil pHRatio (no unit)QTLEcologicalSD0.0513424Ordinal scale
Water qualityWater salinity in surface water (quality of surface water for irrigation)dS/mQTLEcologicalSD0.412223Ordinal scale
Water salinity in groundwater (quality of groundwater for irrigation)dS/mQTLEcologicalSD0.412243Ordinal scale
Arsenic concentration (quality of groundwater for irrigation)ppmQTLEcologicalSD0.222224Ordinal scale
Legend: QNT = Quantitative; QUAL = Qualitative; SD = Secondary data; FO = Field observation.
Table A4. Selected indicators and values to construct single composite indicators for efficiency.
Table A4. Selected indicators and values to construct single composite indicators for efficiency.
Sustainability CategoryComposite IndicatorDescriptionUnitData TypeSustainability PillarData SourceWeightingStudy AreasLevel of Measurement
SSRRIT
EfficiencyMonetary efficiencyMoney input and output in the agro-ecosystem$ output/$ inputQTLEconomicQS11.532.242.786.672.29Ratio scale
Energy efficiencyOverall energy efficiencyRatio of energy output and inputQTLEcologicalQS0.61.372.015.535.545.9Ratio scale
Non-renewable energy efficiencyRatio of energy output and inputQTLEcologicalQS0.40.780.922.172.522.44Ratio scale
Legend: QNT = Quantitative; QS = Questionnaire survey.
Table A5. Selected indicators and values to construct single composite indicators for durability.
Table A5. Selected indicators and values to construct single composite indicators for durability.
Sustainability CategoryComposite IndicatorsDescriptionUnitData TypeSustainability PillarData SourceWeightingStudy AreasLevel of Measurement
SSRRIT
DurabilityResistance to pest stressChemical response to pest stressbinary yes/no responseQUALEcologicalQS0.251.784.174.245.456.54Nominal scale
Water availability at transplanting stage of ricebinary yes/no responseQUALEcologicalQS0.250.750.750.20.20.2Nominal scale
Water availability at flowering stage of ricebinary yes/no responseQUALEcologicalQS0.250.750.750.20.20.2Nominal scale
Farm management (soil test, pest management, land management, soil fertility management)binary yes/no responseQUALEcologicalQS0.250.670.831.691.360.0Nominal scale
Resistance to economic stressGood product pricebinary yes/no responseQUALeconomicQS0.358.4454.584.553.8Nominal scale
Availability of seedsbinary yes/no responseQUALEcologicalQS0.39.339.510108.85Nominal scale
Availability of market (market diversification)binary yes/no responseQUALSocial/economicQS0.35109.178.47107.69Nominal scale
Resistance to climate changeAgricultural trainingbinary yes/no responseQUALSocial/ecologicalQS0.41.331.830.332.271.15Nominal scale
Climate change awarenessbinary yes/no responseQUALSocialQS0.31.110.670.511.820Nominal scale
Advice from agricultural extension workers or NGObinary yes/no responseQUALEcologicalQS0.30.661.170.510.450.38Nominal scale
Legend: QUAL = Qualitative; QS = Questionnaire survey.
Table A6. Selected indicators to construct single composite indicators for compatibility.
Table A6. Selected indicators to construct single composite indicators for compatibility.
Sustainability CategoryComposite IndicatorsDescriptionUnitData TypeSustainability PillarData SourceWeightingStudy AreasLevel of Measurement
SSRRIT
CompatibilityHuman CompatibilityDrinking water quality (protected)binary yes/no responseQUALEcologicalQS0.5089109Nominal scale
Illness from drinking waterbinary yes/no responseQUALEcologicalQS0.5510101010Nominal scale
Biophysical CompatibilityOverall biodiversity condition: percentage of non-crop area%QTLEcologicalQS0.257.546.4823.0115.7318.68Ordinal scale
Overall biodiversity condition: crop richnessnumber of cropsQTLEcologicalQS0.2526161017Ordinal scale
Overall biodiversity condition: crop rotationnumberQTLEcologicalQS0.2523544Ordinal scale
Ecosystem connectivitybinary yes/no responseQUALEcologicalFO0.2511222Nominal scale
Legend: QNT = Quantitative; QUAL = Qualitative; QS = Questionnaire survey; FO = Field observation.
Table A7. Selected indicators and values to construct single composite indicators for equity.
Table A7. Selected indicators and values to construct single composite indicators for equity.
Sustainability CategoryComposite IndicatorsDescriptionUnitData TypeSustainability PillarData SourceWeightingStudy AreasLevel of Measurement
SSRRIT
EquityEducationEducation of farmers%QTLSocialQS0.258.569.254.75105Ordinal scale
Education status of farmers’ male children%QTLSocialQS0.25109.4911.213.17.45Ordinal scale
Education status of farmers’ female children%QTLSocialQS0.259.0710.5411.1712.56.36Ordinal scale
Access to electronic media%QTLSocialQS0.257.789.179.39103.08Ordinal scale
EconomicFarm profitability$QTLEconomicQS0.2648.233340.551371.321992.391025.06Ratio scale
Average wage of farm labourer ($)$/person/dayQTLEconomicQS0.21.331.331.601.801.60Ratio scale
Livelihood diversity other than agricultureCount, 0 to 5QTLEconomicQS0.26.224.335.934.556.92Ordinal scale
Years of economic hardshipNumber of yearsQTLEconomicQS0.20.730.730.910.820.64Ordinal scale
Road network (establishing farm roads and access roads)access/no accessQTLEconomic/socialQS0.223331Nominal scale
HealthSettings where treatment is taken or public health%QTLSocialQS0.53.514.764.078.144.29Ordinal scale
Sanitation or public health%QTLSocialQS0.57.698.737.597.417.08Ordinal scale
GenderWomen’s involvement in decision making about agricultural activities%QTLSocialQS0.53456.52.5Ordinal scale
Gender-based wage differentials$/person/dayQTLEconomicQS0.50.330.330.50.590Ratio scale
Legend: QNT = Quantitative; QS = Questionnaire survey.
Table A8. Results of composite indicators after applying rank normalization and aggregation techniques.
Table A8. Results of composite indicators after applying rank normalization and aggregation techniques.
Results of Rank Normalization and Geometric Mean
CategoryComposite indicatorsSSRRIT
ProductivityProductivity1.002.624.314.642.29
StabilityLandscape stability1.121.411.701.781.82
Soil health/stability1.912.332.241.592.40
Water quality1.001.591.592.002.62
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency1.002.003.004.474.47
DurabilityResistance to pest stress1.682.211.972.001.71
Resistance to economic stress3.423.302.883.171.00
Resistance to climate change3.173.911.823.680.14
CompatibilityHuman compatibility1.002.002.452.832.45
Biophysical compatibility1.191.413.562.713.31
EquityEducation2.452.913.365.001.19
Economic1.742.272.932.931.82
Health2.004.472.453.161.73
Gender1.412.453.464.471.00
Results of Rank Normalization and Arithmetic Mean
CategoryComposite indicatorsSSRRIT
ProductivityProductivity1.002.674.334.672.33
StabilityLandscape stability1.171.501.831.832.00
Soil health/stability2.172.502.501.672.67
Water quality1.001.671.672.332.67
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency1.002.003.004.504.50
DurabilityResistance to pest stress1.752.252.502.502.00
Resistance to economic stress3.673.333.003.331.00
Resistance to climate change3.334.002.004.001.67
CompatibilityHuman compatibility1.002.002.503.002.50
Biophysical compatibility1.251.503.752.753.50
EquityEducation2.503.003.505.001.25
Economic2.002.603.003.002.20
Health2.504.502.503.502.00
Gender1.502.503.504.501.00
Legend: S = Bagda (shrimp)-based agricultural systems (S) from Shyamnagar, SR = Bagda-rice-based agricultural systems (SR) from Kalijang, R = Rice-based agricultural systems (R) from Kalaroa, I = Galda-rice-vegetable-based integrated agricultural systems (I) from Dumuria and T = traditional practices-based agricultural systems (T) from Bhola Sadar. * Only proportionate normalization, no aggregation.
Table A9. Results of composite indicators after applying distance to target normalization and aggregation techniques.
Table A9. Results of composite indicators after applying distance to target normalization and aggregation techniques.
Results of Distance to Target Normalization and Geometric Mean
CategoryComposite indicatorsSSRRIT
ProductivityProductivity0.190.470.890.880.42
StabilityLandscape stability0.510.640.720.760.79
Soil health/stability0.590.810.790.560.83
Water quality0.350.550.550.690.91
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency0.270.350.900.970.98
DurabilityResistance to pest stress0.570.750.460.470.41
Resistance to economic stress0.980.800.770.810.67
Resistance to climate change0.590.670.260.730.14
CompatibilityHuman compatibility#NUM!0.890.951.000.95
Biophysical compatibility0.300.420.980.750.90
EquityEducation0.780.850.761.000.46
Economic0.590.820.790.810.58
Health0.620.760.660.920.65
Gender0.240.270.380.47#NUM!
Results of Distance to Target Normalization and Arithmetic Mean
CategoryComposite indicatorsSSRRIT
ProductivityProductivity0.210.500.890.890.44
StabilityLandscape stability0.540.710.760.790.83
Soil health/stability0.740.820.830.600.86
Water quality0.360.560.560.720.92
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency0.270.350.900.970.98
DurabilityResistance to pest stress0.670.780.550.540.38
Resistance to economic stress0.980.820.800.850.70
Resistance to climate change0.590.720.290.790.28
CompatibilityHuman compatibility0.250.900.951.000.95
Biophysical compatibility0.340.430.990.770.90
EquityEducation0.780.850.791.000.47
Economic0.660.830.830.830.65
Health0.660.790.680.920.67
Gender0.330.350.510.610.04
Legend: S = Bagda (shrimp)-based agricultural systems (S) from Shyamnagar, SR = Bagda-rice-based agricultural systems (SR) from Kalijang, R = Rice-based agricultural systems (R) from Kalaroa, I = Galda-rice-vegetable-based integrated agricultural systems (I) from Dumuria and T = traditional practices-based agricultural systems (T) from Bhola Sadar. * Only proportionate normalization, no aggregation. #NUM! means calculation is not possible.
Table A10. Results of composite indicators after applying Z-score normalization and aggregation techniques.
Table A10. Results of composite indicators after applying Z-score normalization and aggregation techniques.
Results of Z-Score Normalization and Geometric Mean
CategoryComposite indicatorsSSRRIT
ProductivityProductivity#NUM!#NUM!0.960.97#NUM!
StabilityLandscape stability#NUM!#NUM!#NUM!#NUM!#NUM!
Soil health/stability#NUM!#NUM!#NUM!#NUM!#NUM!
Water quality#NUM!#NUM!#NUM!#NUM!1.23
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency#NUM!#NUM!0.630.860.91
DurabilityResistance to pest stress#NUM!#NUM!#NUM!#NUM!#NUM!
Resistance to economic stress#NUM!#NUM!#NUM!#NUM!#NUM!
Resistance to climate change#NUM!#NUM!#NUM!#NUM!0.14
CompatibilityHuman compatibility#NUM!0.330.500.620.50
Biophysical compatibility#NUM!#NUM!1.12#NUM!0.71
EquityEducation#NUM!#NUM!#NUM!1.16#NUM!
Economic#NUM!#NUM!#NUM!#NUM!#NUM!
Health#NUM!#NUM!#NUM!#NUM!#NUM!
Gender#NUM!#NUM!0.641.38#NUM!
Results of Z-Score Normalization and Arithmetic Mean
CategoryComposite indicatorsSSRRIT
ProductivityProductivity−1.29−0.271.031.08−0.54
StabilityLandscape stability0.090.54−0.10−0.06−0.47
Soil health/stability0.84−0.04−0.010.56−1.34
Water quality0.160.78−0.850.78−0.86
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency−1.34−1.080.640.870.91
DurabilityResistance to pest stress0.090.54−0.10−0.06−0.47
Resistance to economic stress0.84−0.04−0.010.56−1.34
Resistance to climate change0.160.78−0.850.78−0.86
CompatibilityHuman compatibility−1.980.360.500.630.50
Biophysical compatibility−1.32−0.941.140.350.77
EquityEducation−0.0270.2970.1121.181−1.563
Economic−0.493−0.0520.5420.424−0.421
Health−0.4480.868−0.3680.708−0.760
Gender−0.468−0.1190.6511.396−1.460
Legend: S = Bagda (shrimp)-based agricultural systems (S) from Shyamnagar, SR = Bagda-rice-based agricultural systems (SR) from Kalijang, R = Rice-based agricultural systems (R) from Kalaroa, I = Galda-rice-vegetable-based integrated agricultural systems (I) from Dumuria and T = traditional practices-based agricultural systems (T) from Bhola Sadar. * Only proportionate normalization, no aggregation. #NUM! means calculation is not possible.
Table A11. Results of composite indicators after applying max-min normalization and aggregation techniques.
Table A11. Results of composite indicators after applying max-min normalization and aggregation techniques.
Results of Max-Min Normalization and Geometric Mean
CategoryComposite indicatorsSSRRIT
ProductivityProductivity#NUM!0.340.840.860.22
StabilityLandscape stability#NUM!#NUM!#NUM!#NUM!#NUM!
Soil health/stability#NUM!#NUM!#NUM!#NUM!#NUM!
Water quality#NUM!#NUM!#NUM!#NUM!0.87
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency#NUM!0.110.860.960.98
DurabilityResistance to pest stress#NUM!0.71#NUM!#NUM!#NUM!
Resistance to economic stress0.750.450.380.54#NUM!
Resistance to climate change0.480.66#NUM!0.450.14
CompatibilityHuman compatibility#NUM!0.890.951.000.95
Biophysical compatibility#NUM!#NUM!0.980.670.84
EquityEducation0.560.65#NUM!1.00#NUM!
Economic#NUM!#NUM!0.680.53#NUM!
Health#NUM!0.520.190.45#NUM!
Gender0.260.460.731.00#NUM!
Results of Max-Min Normalization and Arithmetic Mean
CategoryComposite indicatorsSSRRIT
ProductivityProductivity0.000.380.850.880.27
StabilityLandscape stability0.170.500.540.630.67
Soil health/stability0.670.660.710.370.75
Water quality0.000.280.280.500.89
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency0.000.110.860.960.98
DurabilityResistance to pest stress0.600.750.380.390.25
Resistance to economic stress0.810.490.500.720.00
Resistance to climate change0.490.710.150.700.14
CompatibilityHuman compatibility0.000.900.951.000.95
Biophysical compatibility0.020.150.980.690.85
EquityEducation0.5710.6930.5771.0000.000
Economic0.3470.5010.7920.7170.343
Health0.1850.6350.2150.6000.084
Gender0.3420.4670.7361.0000.000
Legend: S = Bagda (shrimp)-based agricultural systems (S) from Shyamnagar, SR = Bagda-rice-based agricultural systems (SR) from Kalijang, R = Rice-based agricultural systems (R) from Kalaroa, I = Galda-rice-vegetable-based integrated agricultural systems (I) from Dumuria and T = traditional practices-based agricultural systems (T) from Bhola Sadar. * Only proportionate normalization, no aggregation. #NUM! means calculation is not possible.
Table A12. Results of composite indicators after applying proportionate normalization and aggregation techniques.
Table A12. Results of composite indicators after applying proportionate normalization and aggregation techniques.
Results of Proportionate Normalization and Geometric Mean
CategoryComposite indicatorsSSRRIT
ProductivityProductivity0.070.160.300.300.14
StabilityLandscape stability0.140.180.200.210.22
Soil health/stability0.150.210.210.150.22
Water quality0.110.180.180.220.29
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency0.070.160.300.300.14
DurabilityResistance to pest stress0.200.260.160.16#NUM!
Resistance to economic stress0.240.200.190.200.17
Resistance to climate change0.220.250.100.27#NUM!
CompatibilityHuman compatibility#NUM!0.220.240.250.24
Biophysical compatibility0.090.120.290.220.26
EquityEducation0.200.220.200.260.12
Economic0.160.220.210.220.15
Health0.170.210.180.250.18
Gender0.160.190.260.32#NUM!
Results of Proportionate Normalization and Arithmetic Mean (Additive Aggregation)
CategoryComposite indicatorsSSRRIT
ProductivityProductivity0.070.160.300.300.14
StabilityLandscape stability0.140.180.200.210.22
Soil health/stability0.150.210.210.150.22
Water quality0.110.180.180.220.29
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency0.070.160.300.300.14
DurabilityResistance to pest stress0.200.260.160.160.14
Resistance to economic stress0.240.200.190.200.17
Resistance to climate change0.220.250.100.270.14
CompatibilityHuman compatibility0.110.220.240.250.24
Biophysical compatibility0.080.120.300.210.27
EquityEducation0.200.220.200.260.12
Economic0.160.220.210.220.15
Health0.170.210.180.250.18
Gender0.160.190.260.320.12
Legend: S = Bagda (shrimp)-based agricultural systems (S) from Shyamnagar, SR = Bagda-rice-based agricultural systems (SR) from Kalijang, R = Rice-based agricultural systems (R) from Kalaroa, I = Galda-rice-vegetable-based integrated agricultural systems (I) from Dumuria and T = traditional practices-based agricultural systems (T) from Bhola Sadar. * Only proportionate normalization, no aggregation. #NUM! means calculation is not possible.
Table A13. Results of composite indicators after applying weight and multi-criteria aggregation.
Table A13. Results of composite indicators after applying weight and multi-criteria aggregation.
Results of Multi-Criteria Analysis (MCA) Aggregation
CategoryComposite indicatorsSSRRIT
ProductivityProductivity01.403.204.001.20
StabilityLandscape stability1.32.853.253.252.9
Soil health/stability2.42.72.81.43
Water quality3.22.62.63.43.60
EfficiencyMonetary efficiency *0.100.140.180.430.15
Energy efficiency0257.46.40
DurabilityResistance to pest stress2.252.752.52.52.00
Resistance to economic stress3.12.352.252.950.00
Resistance to climate change2.631.33.10.00
CompatibilityHuman compatibility02.53.543.50
Biophysical compatibility0.750.753.752.753.50
EquityEducation1.522.2540.25
Economic1.22.6331.6
Health2.53.51.52.51
Gender1.52340
Legend: S = Bagda (shrimp)-based agricultural systems (S) from Shyamnagar, SR = Bagda-rice-based agricultural systems (SR) from Kalijang, R = Rice-based agricultural systems (R) from Kalaroa, I = Galda-rice-vegetable-based integrated agricultural systems (I) from Dumuria and T = traditional practices-based agricultural systems (T) from Bhola Sadar. * Only proportionate normalization, no aggregation.
Table A14. Productivity: Spearman correlation (in %).
Table A14. Productivity: Spearman correlation (in %).
ProductivityRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM1001050154515
DTTNAM1010030958595
ZSNAM50301008580100
M-MNAM15958510080100
PNAM4585808010080
MCA159510010080100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A15. Landscape stability: Spearman correlation (in %).
Table A15. Landscape stability: Spearman correlation (in %).
Landscape StabilityRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM10085−45858585
DTTNAM85100−8010010060
ZSNAM−45−80100−80−80−50
M-MNAM8510010010010060
PNAM85100−8010010060
MCA8560−5060600100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A16. Soil health/stability: Spearman correlation (in %).
Table A16. Soil health/stability: Spearman correlation (in %).
SOIL HEALTH/STABILITYRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM10085−25759585
DTTNAM85100−809070100
ZSNAM−25−80100−809070
M-MNAM7590−801006090
PNAM9570906010070
MCA8570909070100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A17. Water quality: Spearman correlation (in %).
Table A17. Water quality: Spearman correlation (in %).
Water QualityRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM10010010558510
DTTNAM100100101005585
ZSNAM1010100102555
M-MNAM55100101005585
PNAM8555255510070
MCA1085558570100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A18. Energy efficiency: Spearman correlation (in %).
Table A18. Energy efficiency: Spearman correlation (in %).
Energy EfficiencyRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM1008080807085
DTTNAM801001001003085
ZSNAM801001001003085
M-MNAM801001001003085
PNAM7030303010065
MCA8585858585100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A19. Resistance to pest stress: Spearman correlation (in %).
Table A19. Resistance to pest stress: Spearman correlation (in %).
Resistance to Pest StressRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM100−10−10−103075
DTTNAM−1010090909065
ZSNAM−10901001009065
M-MNAM−10901001009065
PNAM3090909010085
MCA7565656585100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A20. Resistance to economic stress: Spearman correlation (in %).
Table A20. Resistance to economic stress: Spearman correlation (in %).
Resistance to Economic StressRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM10085757510085
DTTNAM85100909085100
ZSNAM75901001007590
M-MNAM75901001007590
PNAM10085757510085
MCA85100909085100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A21. Resistance to climate change: Spearman correlation (in %).
Table A21. Resistance to climate change: Spearman correlation (in %).
Resistance to Climate ChangeRNAMDTTNAMZSNAMM-MNAMPNAMMCA
R.N.A.M10080100807080
D.F.T.N.A.M80100809090100
Z.S.N.A.M10080100807080
M.-M.N.A.M8090801008090
P.N.A.M7090708010090
MCA80100809090100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A22. Human compatibility: Spearman correlation (in %).
Table A22. Human compatibility: Spearman correlation (in %).
Human CompatibilityRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM1001001009555100
DTTNAM1001001009555100
ZSNAM1001001009555100
M-MNAM9595951005095
PNAM5555555010095
MCA1001001009595100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A23. Biophysical compatibility: Spearman correlation (in %).
Table A23. Biophysical compatibility: Spearman correlation (in %).
Biophysical CompatibilityRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM1009510010010095
DTTNAM9510095959590
ZSNAM1009510010010095
M-MNAM1009510010010095
PNAM1009510010010095
MCA9590959595100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A24. Education: Spearman correlation (in %).
Table A24. Education: Spearman correlation (in %).
EducationRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM1008590908055
DTTNAM8510095957540
ZSNAM90951001009045
M-MNAM90951001009045
PNAM8075909010045
MCA5540454545100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A25. Economic: Spearman correlation (in %).
Table A25. Economic: Spearman correlation (in %).
EconomicsRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM10075807080100
DTTNAM7510025358575
ZSNAM8025100905080
M-MNAM7035901006070
PNAM8085506010080
MCA10075807080100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A26. Health: Spearman correlation (in %).
Table A26. Health: Spearman correlation (in %).
HealthRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM1007090906595
DTTNAM7010080801545
ZSNAM90801001002575
M-MNAM90801001002575
PNAM6515252510075
MCA9545757575100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.
Table A27. Gender: Spearman correlation (in %).
Table A27. Gender: Spearman correlation (in %).
GenderRNAMDTTNAMZSNAMM-MNAMPNAMMCA
RNAM10095100100100100
DTTNAM9510095959595
ZSNAM10095100100100100
M-MNAM10095100100100100
PNAM10095100100100100
MCA10095100100100100
Legend: RNAM = Rank normalization and arithmetic mean; DTTNAM = Distance to target normalization and arithmetic mean; ZSNAM = Z-score normalization and arithmetic mean; M-MNAM = Max-Min normalization and arithmetic mean; PNAM = Proportionate normalization and arithmetic mean; MCA = Multi-criteria analysis.

References

  1. De Muro, P.; Mazziotta, M.; Pareto, A. Composite indices for multidimensional development and poverty: An application to MDG indicators. In Proceedings of the Wye City Group Meeting, Rome, Italy, 11–12 June 2009; Available online: http://www.fao.org/fileadmin/templates/ess/pages/rural/wye_city_group/2009/Paper_3_b2_DeMuro-Mazziotta-Pareto_Measuring_progress_towards_MDGs.pdf (accessed on 10 May 2015).
  2. Hayati, D.; Ranjbar, Z.; Karami, E. Measuring agricultural sustainability. In Biodiversity, Biofuels, Agroforestry and Conservation Agriculture; Springer: Dordrecht, The Netherlands, 2011; pp. 73–100. [Google Scholar]
  3. Food and Agriculture Organization of the United Nations (FAO). Sustainability Assessment of Food and Agriculture Systems (SAFA); FAO: Rome, Italy, 2012. [Google Scholar]
  4. Falcone, G.; De Luca, A.I.; Stillitano, T.; Strano, A.; Romeo, G.; Gulisano, G. Assessment of Environmental and Economic Impacts of Vine-Growing Combining Life Cycle Assessment, Life Cycle Costing and Multicriterial Analysis. Sustainability 2016, 8, 793. [Google Scholar] [CrossRef]
  5. De Luca, A.I.; Iofrida, N.; Leskinen, P.; Stillitano, T.; Falcone, G.; Strano, A.; Gulisano, G. Life cycle tools combined with multi-criteria and participatory methods for agricultural sustainability: Insights from a systematic and critical review. Sci. Total Environ. 2017, 595, 352–370. [Google Scholar] [CrossRef] [PubMed]
  6. Van Cauwenbergh, N.; Biala, K.; Bielders, C.; Brouckaert, V.; Franchois, L.; Cidad, V.G.; Hermy, M.; Mathijs, E.; Muys, B.; Reijinders, J.; et al. SAFE—A hierarchical framework for assessing the sustainability of agricultural systems. Agric. Ecosyst. Environ. 2007, 120, 229–242. [Google Scholar] [CrossRef]
  7. VanLoon, G.W.; Patil, S.G.; Hugar, L.B. Agricultural Sustainability: Strategies for Assessment; Sage: New Delhi, India, 2005. [Google Scholar]
  8. Gómez-Limón, J.A.; Sanchez-Fernandez, G. Empirical evaluation of agricultural sustainability using composite indicators. Ecol. Econ. 2010, 69, 1062–1075. [Google Scholar] [CrossRef]
  9. Andreoli, M.; Tellarini, V. Farm sustainability evaluation: Methodology and practice. Agric. Ecosyst. Environ. 2000, 77, 43–52. [Google Scholar] [CrossRef]
  10. Pirazzoli, C.; Castellini, A. Application of a model for evaluating the environmental sustainability of cultures in hill and mountain areas. The case of berries and fruit chestnut in Northern Italy. Agric. Econ. Rev. 2000, 1, 57–70. [Google Scholar]
  11. Rigby, D.; Woodhouse, P.; Young, T.; Burton, M. Constructing a farm level indicator of sustainable agricultural practice. Ecol. Econ. 2001, 39, 463–478. [Google Scholar] [CrossRef]
  12. Van Calker, K.J.; Berentsen, P.B.M.; Romero, C.; Giesen, G.W.J.; Huirne, R.B.M. Development and application of a multi-attribute sustainability function for Dutch dairy farming systems. Ecol. Econ. 2006, 57, 640–658. [Google Scholar] [CrossRef]
  13. The Organisation for Economic Co-operation and Development (OECD). Handbook on Constructing Composite Indicators. Methodology and User Guide; Joint Research Centre-European Commission: Paris, France, 2008. [Google Scholar]
  14. Rasul, G.; Thapa, G.B. Sustainability analysis of ecological and conventional agricultural systems in Bangladesh. World Dev. 2003, 31, 1721–1741. [Google Scholar] [CrossRef]
  15. Roy, R.; Chan, N.W.; Rainis, R. Rice farming sustainability assessment in Bangladesh. Sustain Sci. 2014, 9, 31–44. [Google Scholar] [CrossRef]
  16. Talukder, B.; Saifuzzaman, M.; vanLoon, G.W. Sustainability of agricultural systems in the coastal zone of Bangladesh. Renew. Agric. Food Syst. 2016, 31, 148–165. [Google Scholar] [CrossRef]
  17. Sumner, A.; Tezanos Vázquez, S. How Has the Developing World Changed Since the Late 1990s? A Dynamic and Multidimensional Taxonomy of Developing Countries; Working Paper 375; Center for Global Development: Washington, DC, USA, 2014. [Google Scholar]
  18. Foa, R.; Tanner, J. Methodology of the Indices of Social Development; ISD Working Paper Series, No. 2012-04; Indices of Social Development, 2012; Available online: https://repub.eur.nl/pub/50510/ (accessed on 15 June 2015).
  19. Benini, A. Composite Measures: Their Use in Rapid Needs Assessments: Conceptual Background and Technical Guidance. 2012. Version 17. Available online: http://aldo-benini.org/Level2/HumanitData/ACAPS_CompositeMeasures_Note_120517AB.pdf (accessed on 16 June 2015).
  20. Nardo, M.; Saisana, M.; Saltelli, A.; Tarantola, S. Tools for Composite Indicators Building; European Commission, Institute for the Protection and Security of the Citizen, JRC: Ispra, Italy, 2005. [Google Scholar]
  21. Saisana, M.S.A.; Saltelli, A. Rankings and ratings: Instructions for use. Hague J. Rule Law 2011, 3, 247–268. [Google Scholar] [CrossRef]
  22. Cherchye, L.; Moesen, W.; Rogge, N.; Van Puyenbroeck, T. An introduction to ‘benefit of the doubt’ composite indicators. Soc. Indic. Res. 2007, 82, 111–145. [Google Scholar] [CrossRef]
  23. Sharpe, A. Literature Review of Frameworks for Macro-Indicators; No. 2004-03; Centre for the Study of Living Standards: Ottawa, ON, Canada, 2004. [Google Scholar]
  24. Saisana, M.; Tarantola, S.; Schulze, N.; Cherchye, L.; Moesen, W.; Van Puyenbroeck, T. State-of-the-Art Report on Composite Indicators for the Knowledge-Based Economy. Work-Package 5. 2005. Available online: https://www.uni-trier.de/fileadmin/fb4/projekte/SurveyStatisticsNet/KEI-WP5-D5.1.pdf (accessed on 15 August 2016).
  25. Munda, G.; Saisana, M. Methodological considerations on regional sustainability assessment based on multicriteria and sensitivity analysis. Reg. Stud. 2011, 45, 261–276. [Google Scholar] [CrossRef]
  26. Booysen, F. An overview and evaluation of composite indices of development. Soc. Indic. Res. 2002, 59, 115–151. [Google Scholar] [CrossRef]
  27. Salzman, J. Methodological Choices Encountered in the Construction of Composite Indices of Economic and Social Well-Being; Center for the Study of Living Standards: Ottawa, ON, Canada, 2003. [Google Scholar]
  28. Sajeva, M.; Gatelli, D.; Tarantola, S.; Hollanders, H. Methodology Report on European Innovation Scoreboard 2005; European Commission: Brussels, Belgium, 2005. [Google Scholar]
  29. Freudenberg, M. Composite Indicators of Country Performance: A Critical Assessment; OECD Science, Technology and Industry Working Papers, 2003/16; OECD Publishing: Paris, France, 2003. [Google Scholar]
  30. Hudrlíková, L.; Kramulová, J. Do transformation methods matter? The case of sustainability indicators in Czech regions. Adv. Methodol. Stat. 2013, 10, 31–48. [Google Scholar]
  31. Ebert, U.; Welsch, H. Meaningful environmental indices: A social choice approach. J. Environ. Econ. Manag. 2004, 47, 270–283. [Google Scholar] [CrossRef]
  32. Tate, E. Social vulnerability indices: A comparative assessment using uncertainty and sensitivity analysis. Nat. Hazards 2012, 63, 325–347. [Google Scholar] [CrossRef]
  33. Talukder, B. Sustainability of Changing Agricultural Systems in the Coastal Zone of Bangladesh. Master’s Thesis, School of Environmental Studies, Queen’s University, Kingston, ON, Canada, 2012. [Google Scholar]
  34. Blanc, I.; Friot, D.; Margni, M.; Jolliet, O. Towards a new index for environmental sustainability based on a DALY weighting approach. Sustain. Dev. 2008, 16, 251–260. [Google Scholar] [CrossRef]
  35. Nardo, M.; Saisana, M.; Saltelli, A.; Tarantola, S.; Hoffman, H.; Giovannini, E. Handbook on Constructing Composite Indicators: Methodology and User Guide; Organisation for Economic Co-operation and Development: Paris, France, 2009. [Google Scholar]
  36. Pomerol, J.C.; Barba-Romero, S. Multicriterion Decision in Management: Principles and Practice; Springer Science & Business Media: Berlin, Germany, 2012; Volume 25. [Google Scholar]
  37. Tofallis, C. Add or multiply? A tutorial on ranking and choosing with multiple criteria. INFORMS Trans. Educ. 2014, 14, 109–119. [Google Scholar] [CrossRef]
  38. Mitchell, H.B. Data Fusion: Concepts and Ideas; Springer Science & Business Media: Berlin, Germany, 2012. [Google Scholar]
  39. Jacobs, R.; Smith, P.; Goddard, M.K. Measuring Performance: An Examination of Composite Performance Indicators: A Report for the Department of Health; Centre of Health Economics, University of York: York, UK, 2004; Available online: http://www.york.ac.uk/che/pdf/tp29.pdf (accessed on 10 June 2016).
  40. Jencks, S.F.; Huff, E.D.; Cuerdon, T. Change in the quality of care delivered to Medicare beneficiaries, 1998–1999 to 2000–2001. J. Am. Med. Assoc. 2003, 289, 305–312. [Google Scholar] [CrossRef]
  41. Saisana, M.; Tarantola, S. State-of-the-Art Report on Current Methodologies and Practices for Composite Indicator Development; Institute for the Protection and the Security of the Citizen Technological and Economic Risk Management Unit: Ispra, Italy, 2002. [Google Scholar]
  42. Mei, Z.; Grummer-Strawn, L.M. Standard deviation of anthropometric Z-scores as a data quality assessment tool using the 2006 WHO growth standards: A cross country analysis. Bull. World Health Organ. 2007, 85, 441–448. [Google Scholar] [CrossRef] [PubMed]
  43. World Bank (WB). KEI and KI Indexes (KAM). 2009. Available online: http://info.worldbank.org/etools/kam2/kam_page5.asp (accessed on 1 August 2014).
  44. De Onis, M. WHO Child Growth Standards: Length/Height-for-Age, Weight-for-Age, Weight-for-Length, Weight-for-Height and Body Mass Index-for-Age; World Health Organization (WHO): Geneva, Switzerland, 2006. [Google Scholar]
  45. United Nations Development Programme (UNDP). Human Development Index (HDI) Report 2014; UNDP: New York, NY, USA, 2014. [Google Scholar]
  46. Dailey, G. Normalizing Census Data in ArcView. ArcUser Online. 2000, pp. 44–45. Available online: https://www.esri.com/news/arcuser/1000/files/normalize.pdf (accessed on 11 March 2015).
  47. Decancq, K.; Lugo, M.A. Setting Weights in Multidimensional Indices of Well-Being. 2008. Available online: http://www.ophi.org.uk/wp-content/uploads/ssDecancq-Lugo-2008.pdf (accessed on 16 January 2014).
  48. Sen, A.K.; Foster, J. On Economic Inequality; Oxford University Press: Oxford, UK, 1997. [Google Scholar]
  49. United Nations Development Programme (UNDP). Human Development Index (HDI) Report 2011; UNDP: New York, NY, USA, 2011. [Google Scholar]
  50. Saltelli, A. Composite indicators between analysis and advocacy. Soc. Indic. Res. 2007, 81, 65–77. [Google Scholar] [CrossRef]
  51. Tarabusi, E.C.; Guarini, G. An unbalance adjustment method for development indicators. Soc. Indic. Res. 2013, 112, 19–45. [Google Scholar] [CrossRef]
  52. Mazziotta, M.; Pareto, A. Comparing two non-compensatory composite indices to measure changes over time: A case study. Statistika 2015, 95, 44–53. Available online: https://www.czso.cz/documents/10180/20550317/32019715q2044.pdf/693ab726-62b8-4163-8569-f47b4841245d?version=1.0 (accessed on 7 June 2016).
  53. Munda, G.; Nardo, M. Constructing Consistent Composite Indicators: The Issue of Weights; EUR 21834 EN; European Commission: Brussels, Belgium, 2005. [Google Scholar]
  54. European Commission (EC). Tool #57: Multi-Criteria Analysis. 2005. Available online: http://ec.europa.eu/smart-regulation/guidelines/tool_57_en.htm (accessed on 15 July 2015).
  55. Brand, D.A.; Saisana, M.; Rynn, L.A.; Pennoni, F.; Lowenfels, A.B. Comparative analysis of alcohol control policies in 30 countries. PLoS Med. 2007, 4, e151. [Google Scholar] [CrossRef] [PubMed]
  56. Saltelli, A.; Nardo, M.; Saisana, M.; Tarantola, S. Composite indicators: The controversy and the way forward. In Statistics, Knowledge and Policy Key Indicators to Inform Decision Making: Key Indicators to Inform Decision Making; The Organisation for Economic Co-operation and Development (OECD): Paris, France, 2005; pp. 359–372. [Google Scholar]
  57. Stano, P. Aggregation Rules (I). Compensatory Approach. In Proceedings of the 12th JRC Annual Training on Composite Indicators & Multicriteria Decision Analysis (COIN 2014), Ispra, Iatly, 22–26 September 2014; European Commission Joint Research Centre Econometrics, Applied Statistics Unit Composite Indicators Research Group (JRC-COIN). Available online: https://ec.europa.eu/jrc/sites/default/files/2014092226COINAgenda.pdf (accessed on 9 June 2015).
Figure 1. Generalized graphical representation of normalization for constructing a composite indicator.
Figure 1. Generalized graphical representation of normalization for constructing a composite indicator.
Resources 06 00066 g001
Figure 2. Scheme for the construction and evaluation process for single composite indicators. Legend: Pro = Productivity; LS = Landscape stability; SH/S = Soil health/stability; WQ = Water quality; ME = Monetary efficiency; EE = Energy efficiency; RTPS = Resistance to pest stress; RTES = Resistance to economic stress; RTCC = Resistance to climate change; HC = Human compatibility; BC = Biophysical compatibility; Edu = Education; Eco = Economic; Heal = Health; Gen = Gender; Pptn = Proportionate; Ran = Ranking; DTT = Distance to target; CS = Categorical scale; Min-Max = Min-max technique; ZS = Z-score. Source: Compiled by the authors.
Figure 2. Scheme for the construction and evaluation process for single composite indicators. Legend: Pro = Productivity; LS = Landscape stability; SH/S = Soil health/stability; WQ = Water quality; ME = Monetary efficiency; EE = Energy efficiency; RTPS = Resistance to pest stress; RTES = Resistance to economic stress; RTCC = Resistance to climate change; HC = Human compatibility; BC = Biophysical compatibility; Edu = Education; Eco = Economic; Heal = Health; Gen = Gender; Pptn = Proportionate; Ran = Ranking; DTT = Distance to target; CS = Categorical scale; Min-Max = Min-max technique; ZS = Z-score. Source: Compiled by the authors.
Resources 06 00066 g002
Figure 3. Location of the study areas and gradients of soil salinity (1973–2009) in the coastal zone of Bangladesh. The soil salinity contours represent the northern boundary of areas where soils may have salinity values of 2 dS m−1 or more ([16], p. 149).
Figure 3. Location of the study areas and gradients of soil salinity (1973–2009) in the coastal zone of Bangladesh. The soil salinity contours represent the northern boundary of areas where soils may have salinity values of 2 dS m−1 or more ([16], p. 149).
Resources 06 00066 g003
Table 1. Selected normalization techniques for this study.
Table 1. Selected normalization techniques for this study.
Name FormulaExplanation
Ranking [35] N ias = Rank ( X ias ) Where N ias = normalized value of indicator i for agricultural systems , X ias = variable X for indicator i for agricultural systems as
Distance to target [35] N ias = X ias Target   X ias   Where N ias = normalized value of indicator i for agricultural systems as , X ias = variable X for indicator i for agricultural systems as
Z-score (Standardization) [35] N ias = ( X ias µ ) / σ Where N ias = normalized value of indicator i for agricultural systems as , X ias = variable X for indicator i for agricultural systems as , µ (Mu) = mean of indicator values, σ (Sigma) = standard deviation (square root of the variance) of indicators
Min-max [35] N ias = X ias min as ( X i ) max as ( X ias )   min as ( X i ) Where   N ias = normalized value of indicator i for agricultural systems as , X i = indicator, X ias = variable X for indicator i for agricultural systems as ; max as and min as are the largest and smallest observed values
Proportionate [36] N ias = I i i I i   0 < N ias < 1Where I i = indicator value,   i I i = sum of the indicators
Table 2. Composite indicators developed using proportionate normalization and hybrid aggregation techniques.
Table 2. Composite indicators developed using proportionate normalization and hybrid aggregation techniques.
Sustainability CategoriesIndicatorsAgricultural SystemsAggregation Technique/Comments
SSRRIT
ProductivityProductivity0.070.160.300.300.14GM
StabilityLandscape stability0.140.180.200.210.22GM
Soil health/stability0.150.210.210.150.22GM
Water quality0.110.180.180.220.29GM
EfficiencyMonetary efficiency0.100.140.180.430.15Only normalized
Energy efficiency0.070.160.300.300.14GM
DurabilityResistance to pest stress0.240.270.190.180.12AM
Resistance to economic stress0.250.200.190.200.17AM
Resistance to climate change0.220.270.110.300.10AM
CompatibilityHuman compatibility0.060.220.240.250.24AM
Biophysical compatibility0.100.130.290.220.27AM
EquityEducation0.200.220.200.260.12AM
Economic0.170.230.210.220.17AM
Health0.170.210.180.260.18AM
Gender0.170.190.260.320.06AM
Legend: GM = Geometric Mean, AM = Arithmetic Mean. Note: Monetary efficiency was not normalized or aggregated as original data are used for composite indicator values. S = Bagda (shrimp)-based agricultural systems (S) from Shyamnagar, SR = Bagda-rice-based agricultural systems (SR) from Kalijang, R = Rice-based agricultural systems (R) from Kalaroa, I = Galda-rice-vegetable-based integrated agricultural systems (I) from T = Dumuria and traditional practices-based agricultural systems (T) from Bhola Sadar.

Share and Cite

MDPI and ACS Style

Talukder, B.; W. Hipel, K.; W. vanLoon, G. Developing Composite Indicators for Agricultural Sustainability Assessment: Effect of Normalization and Aggregation Techniques. Resources 2017, 6, 66. https://doi.org/10.3390/resources6040066

AMA Style

Talukder B, W. Hipel K, W. vanLoon G. Developing Composite Indicators for Agricultural Sustainability Assessment: Effect of Normalization and Aggregation Techniques. Resources. 2017; 6(4):66. https://doi.org/10.3390/resources6040066

Chicago/Turabian Style

Talukder, Byomkesh, Keith W. Hipel, and Gary W. vanLoon. 2017. "Developing Composite Indicators for Agricultural Sustainability Assessment: Effect of Normalization and Aggregation Techniques" Resources 6, no. 4: 66. https://doi.org/10.3390/resources6040066

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop