Next Article in Journal
Experimental Evaluation of a Diesel Cogeneration System for Producing Power and Drying Aromatic Herbs
Next Article in Special Issue
Ammonia Volatilization Losses during Irrigation of Liquid Animal Manure
Previous Article in Journal
Welfare Economic Analysis of Lifting Water Subsidies for Banana Farms in Jordan
Previous Article in Special Issue
One Concept, Many Opinions: How Scientists in Germany Think About the Concept of Bioeconomy
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Agricultural Sustainability: A Review of Concepts and Methods

Maria G. Lampridi
Claus G. Sørensen
3 and
Dionysis Bochtis
Center for Research & Technology Hellas—CERTH, Institute for Bio-economy and Agri-technology—iBO, 10th km Thessalonikis—Thermis, BALKAN Center, BLDG D, 57001 Thessaloniki, Greece
Faculty of Agriculture, Forestry and Natural Environment, School of Agriculture, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece
Department of Engineering, Aarhus University, Finlandsgade 22, DK8200 Aarhus N, Denmark
Author to whom correspondence should be addressed.
Sustainability 2019, 11(18), 5120;
Submission received: 21 August 2019 / Revised: 11 September 2019 / Accepted: 18 September 2019 / Published: 19 September 2019
(This article belongs to the Special Issue Green, Closed Loop, Circular Bio-Economy)


This paper presents a methodological framework for the systematic literature review of agricultural sustainability studies. The framework synthesizes all the available literature review criteria and introduces a two-level analysis facilitating systematization, data mining, and methodology analysis. The framework was implemented for the systematic literature review of 38 crop agricultural sustainability assessment studies at farm-level for the last decade. The investigation of the methodologies used is of particular importance since there are no standards or norms for the sustainability assessment of farming practices. The chronological analysis revealed that the scientific community’s interest in agricultural sustainability is increasing in the last three years. The most used methods include indicator-based tools, frameworks, and indexes, followed by multicriteria methods. In the reviewed studies, stakeholder participation is proved crucial in the determination of the level of sustainability. It should also be mentioned that combinational use of methodologies is often observed, thus a clear distinction of methodologies is not always possible.

1. Introduction

The world’s population is rapidly increasing and, according to the most recent projections, it is expected to reach 9.8 million in 2050 and 11.2 million in 2100 [1]. To that end, the planet should be ready to cope with the expected rapid population growth. Producing and delivering adequate, high quality food will be one of the most important challenges for humanity in the next century [2]. The evolution of technology has led to intensification of agricultural production leading to increased productivity and (in most of the cases) quality of agriproducts as well. However, this intensification has significantly increased the environmental footprint of agriculture, leading to a number of environmental impacts associated with the extensive use of fertilizers, pesticides, water, changes in land use, etc. [3]. The environmental issues related to agriculture have drawn the attention of the scientific community, which is now turning towards exploring the definition of agricultural sustainability without having yet reached consensus [4,5].
Undoubtingly, defining agricultural sustainability, as with every other sustainability concept, is a challenging task. Nevertheless, it is a common agreement that agricultural sustainability should at least address the three basic pillars of sustainable development by appraising simultaneously environmental, economic, and social issues related to agricultural practices [6]. However, the sustainability assessment of agricultural practices, in general, can be a very challenging task since it involves many case-specific variables to be taken under consideration.
Figure 1 presents various processes, inputs, and outputs involved in agricultural production, demonstrating the difficulty and complexity in generalizing the sustainability assessment process. There are general cultivation guidelines and corresponding operations stages for almost all crops (e.g. seeding, irrigation, and harvesting). However, the agronomic practice, the machinery types, the technology level, as well as the quantities and type of materials used may vary, depending on the type of crop, the implementation practice, the country (even the region of the cultivation), and the prevailing climatic conditions. All of the aforementioned parameters affect the cultivation process and the respective inflows and outflows.
It is obvious that the standardization of the Agricultural Sustainability Assessment is a challenging task. Considering the growing interest in assessing the sustainability issues related to agriculture, several tools and methodologies have been developed [7,8]. Among those tools some have gained greater acceptance and are widely used by the majority of practitioners worldwide, such as life cycle assessment (LCA), which is standardized by ISO in ISO 14040:2006 and ISO 14044:2006 [9]. In addition, many indicator-based methods have been developed for the sustainability assessment of agricultural practices that use different approaches with regards to the overall objective, the intended users, and the definition of agricultural sustainability they employ [4].
Considering what was mentioned above and that there is not yet an established standardized methodology, it is very important for anyone attempting to assess agricultural sustainability to have an overview of the available and most usually used methodologies and tools to that scope. As a result, there is a need for a methodological framework that will help practitioners to evaluate the existing available tools and methods in order to select the appropriate one, for each specific task.
To that end, the present paper has a two-fold objective:
  • To determine the evaluation criteria to systematically review agricultural sustainability assessment studies. To that end, several review papers were selected based on specific selection criteria and examined to determine the goal as well as the individual evaluation criteria adopted in each review. The ultimate goal is to critically synthesize a methodological framework for the systematic recording and evaluation of available agricultural sustainability assessment studies. Such systematic documentation can facilitate the comparison among the available studies as well as the development of a standard methodological framework for the sustainability assessment of agriculture.
  • To implement the proposed methodology by investigating the available and mostly used methodologies to assess the sustainability of crop cultivations at the farm level. The methodological framework is applied to 38 Agricultural Sustainability studies published in peer-reviewed journals in the last decade (2009–2018).

2. Materials and Methods

2.1. Methodological Framework for the Systematic Review of Agricultural Sustainability Studies

2.1.1. Research Design

The evaluation process implemented to assess and select the criteria needed for the methodological framework of the systematic review on agricultural sustainability studies is presented in Figure 2. Initially, scientific literature published in Science Direct and Scopus was searched using the specific keywords and Boolean operators (AND/OR). The keywords were selected with respect to the integrated concept of “sustainability assessment”, as well as the individual processes it consists of, namely, “environmental assessment”, “economic assessment”, and “societal assessment” (or “social assessment”) combined with the keywords agriculture/farming using the Boolean Operator AND to exclude results that are not relevant to the field under examination. It should be added that the concept of “agricultural sustainability” was also included in the search.
The first sample of scientific papers that resulted from the initial search included 55 papers from peer-reviewed scientific journals. These papers were put through a screening process considering specific exclusion criteria presented in Figure 2. Specifically, studies that were not related to agriculture and especially focused on alternative agricultural processes were excluded. As a result, papers exclusively focused on aquaculture or organic farming studies, biofuels and biorefinery as well as review of studies comparing agronomic protocols were excluded from the present assessment. Additionally, review studies regarding soil quality, land management, food processing systems and discussions that did not specifically define the methods of the review conducted, were excluded. At this point, it should also be stated that in the context of agricultural sustainability studies, livestock farming was included in the search.
The final paper collection comprises 16 review papers or studies that assess agricultural sustainability. It should be noted that the literature is relatively scarce regarding studies that consider all the three dimensions of sustainability with respect to other scientific fields, for example, the secondary production of goods. To that end, the sample includes studies considering the environmental aspect of agricultural sustainability which is the most often studied. The sample was then assessed in two ways, a systematic and critical [10]. The systematic way concerns the listing of the papers based on specifically defined criteria [11]. The initial listing criteria in the case of the presented framework, include the title and author of the paper, the year of publication as well as the spatial coverage of the study (Global or Regional) and the type of review (Critical or Systematic).
Critical reviews are thorough literature works that attempt to evaluate and assess the basic aspects or inputs and document the differences in methodology and implementation of scientific studies on a specific field [11]. In this case, the critical evaluation of the sample concerns the individual analysis of the selected studies with the purpose of extracting the individual evaluation criteria used in each study. The individual criteria with similar context were aggregated in a general table of criteria. Then, each paper was systematically reviewed as to whether each criterion was included in the review.
The resulting table is a comprehensive overview of the issues most frequently examined in a review study. The criteria that were used the most are the criteria that should be integrated in the methodological framework for the systematic review of agricultural sustainability studies. The rule followed in the present paper was to exclude criteria that were used in less than four papers. Following next is the sample presentation as well as the criteria frequency table along with a critical assessment of the sample used for the evaluation.

2.1.2. Systematic Approach

The 16 review papers that were extracted by the implementation of the first steps of the methodology, presented in the previous section are presented in Table 1 along with their classification with respect to their type and spatial coverage.
As presented in Figure 3, during 2016–2017, the number of review papers has increased, indicating a boosted interest in the sustainability of agricultural practices. Payraudeau et al. (2005) first analyzed and systematically reviewed six (6) agricultural sustainability methods employed in eleven (11) case studies, indicating the variety of objectives, target groups, and methodologies used [20]. Bockstaller et al. (2008), followed by presenting a typology of indicators and the evolution of the methods used for their advancement [19], in 2009, critically evaluateing four (4) comparative studies to analyze the methods of the comparison, highlighting their main results [23]. Also focusing on indicators, Binder et al. (2010) presented an evaluation review framework that was used to review agricultural sustainability methods [4]. The framework assessed the normative, systematic and procedural aspects of the methods under evaluation.
Regarding the types of review papers and their classification to systematic or critical according to the definitions presented in the previous section [11], it is observed that, in principal, both categories are equally preferred by the researchers. However, in some cases, the distinction is not clear or a systematic and critical review is performed at the same time. Such example is the work of De Luca et al. (2017), where authors performed a critical and systematic review to determine, among other issues, which Multi Criteria Decision Analysis (MCDA) and participatory methods have been used along with LCA tools and the type of integration used in each case [10]. Also, Baldini et al. (2017) critically reviewed forty-four (44) LCA studies on milk production and systematically compared their methods and results to highlight issues requiring further discussion and investigation [15].
Considering the selected samples, it can be stated that in most cases systematic reviews are used in order to compare methodologies and results regarding a specific field of agricultural application. Towards this objective, Peter et al. (2017) performed a systematic evaluation of eighteen (18) carbon footprint calculators used in energy crop cultivations [12]. Cerutti et al. (2011) systematically reviewed twenty-two (22) fruit production sustainability assessment studies [7], whereas De Vries et al. (2015) systematically reviewed LCA studies on beef production [21]. Additionally, McAuliffe et al. (2015) conducted a chronological review of LCA studies in pig production, attempting to demonstrate how LCA has captured technological advancements in the field as well as the methodological issues observed [17].
On the contrary, the majority of the reviews that were characterized as critical are dealing with the evaluation of indicator-based methods or the classification of agricultural sustainability indicators, such as the work of Acosta-Alba et al. (2011), who reviewed eight (8) agricultural sustainability frameworks that use reference values for their indicators and analyzed the methods for the establishment of the reference values and investigating ways for their improvement [14]. Latruffe et al. (2016) provided a review of the available agricultural sustainability indicators, highlighting the relative high increase of environmental indicators as compared with the smaller interest in economic and social indicators [18]. Finally, Lebacq et al. (2013) reviewed the types of sustainability indicators and proposed indicative ground rules for the selection of agricultural sustainability indicators [22].
With respect to the spatial coverage of the reviews (Figure 4), the majority deals with studies from all around the world. Nevertheless, there are reviews assessing studies in specific countries or regions. For example, Roy et al. (2012), based on a systematic review and synthesis, presents a set of indicators that could be used to assess agricultural sustainability in Bangladesh, highlighting the need for integrated approaches and participatory processes during agricultural sustainability assessment [13]. Additionally, Morais et al. (2016) systematically reviewed twenty-two (22) agri-food-dedicated LCA studies in Portugal, revealing issues regarding the challenges faced and the lack of systematic regional approach in the country that could safeguard the accuracy and comparability of the results [16]. Lastly, Yan et al. (2011) reviewed thirteen (13) LCA studies on European milk production, indicating that direct comparison is challenging due to inconsistency regarding the used methodologies [9].

2.1.3. Critical Approach

The selected sample, which was thoroughly described in the previous section, was screened, to extract the individual evaluation criteria used during each review. As some criteria had the same objective or were of the same context they were categorized accordingly. Also, some studies further analyzed the criteria including various subcriteria, but this is out of the scope of this paper since it is an issue related to the scrutiny of the review each author aims to achieve and the corresponding scope.
Figure 5 presents the criteria identified during the screening process and the frequency of their occurrence. A total of forty-four (44) different criteria were used in the sixteen (16) studies reviewed. The review criteria frequency table is presented in detail in Appendix A (Table A1). The first six criteria (beginning from the top of Figure 5) were common in most of the reviews examined and include the name and description of the assessment method or tool, the field of application, the country of application, and the year of issuing. The literature typology concerns the type of the document reviewed. For example, De Luca et al. (2017) classified the selected publications into three categories (Journal Article, Book Chapter, and Conference Proceedings paper) [10]. Baldini et al. (2017), on the other hand, refers to publication types classifying the sample according to whether the literature is an original article, a review, a research direction, or a scenario analysis [15].
Regarding the level of assessment and the system boundaries, many different approaches were identified. Payraudeau et al. (2005), Roy et al. (2012), and de Luca et al. (2017) assessed the level of assessment of the studies reviewed regarding its spatial coverage (global, national, regional, local, and farm level) [10,13,20]. Lebacq et al. (2013) distinguished studies by whether they concern a farm or a region, and Bockstaller et al. (2009) examined the level of assessment giving two different scale options (field scale or farm scale) [22,23]. Cerutti et al. (2011) and Baldini et al. (2017) identified the system boundaries of the studies assessed [7,15], following a cradle-to-gate or cradle-to-market approach, whereas de Vries et al. (2015) reviewed studies at least from cradle-to-farm gate [21]. Lastly, Peter et al. (2017) examined both the level of assessment (global, regional, etc.) and the system boundaries (farm–gate or farm–gate–grave) of the studies they review [12].
The issue of the intended user of a method or tool is being considered in several of the studies reviewed. Binder et al. (2010) identified the target group of the examined methodologies [4], whereas de Luca et al. (2017) referred to the specific criterion as actors involved in the assessment process (i.e., local experts, scientists, workers, etc.) [10]. Bockstaller et al. (2008) classified the reviewed works according to the target user of the method reviewed, i.e. decision-maker, researcher, technician or farmer [19]. Considering the type and the accessibility of data criteria, Baldini et al. (2017) distinguish the data in experimental and model data [15]. The accessibility of data (or availability as expressed by Roy et al. 2012 [13]) is examined by Bockstaller et al. (2008) for three user groups, farmers, advisors, and administration [19].
With reference to the name and type of the indicators reviewed, many approaches were identified during the screening process. Lebacq et al. (2013) and Latruffe et al. (2016) categorized indicators based on their representation of the three pillars of sustainability (environmental, economic and social) [18,22]. Lebacq et al. (2013), however, extended the research scope by identifying means-based, system state, emission, and effect-based indicators [22]. Based on the calculation method Latruffe et al. (2016), Lebacq et al. (2013) and Bockstaller et al. (2008) distinguish indicators based on the method used for their calculation, i.e., single variables, emission factors, combination of variables, operational models, mechanistic models, etc. [18,19,22]. De Luca et al. (2017) categorize LCA indicators based on the impact categories they address [10].
Based on the rule set in the methodology section, the red line in Figure 5 presents the criteria exclusion threshold. Only criteria identified more than four times in the sample reviewed are included in the methodological approach for the systematic review of agricultural sustainability studies. A total of eighteen (18) criteria surpassed the exclusion threshold. These criteria are classified in groups with respect to their context and are presented in the subsequent section.

2.1.4. Methodological Framework Presentation

Following the criteria determination process described in the previous sections, Figure 6 presents the critical synthesis to systematically review agricultural sustainability related studies. The proposed methodological framework is based on a series of criteria and divided into five (5) underlying categories. The first two categories refer to the initial screening stage. During this preliminary stage, the studies are assessed to determine if the study will be included in the sample on the basis of the case-specific exclusion criteria determined with regards to the scope of the review.
The initial screening stage includes two categories (i.e., “method identification” and “general information”) of criteria with respect to the basic description of each study. The general information of a study concerns the year of publication and the type of literature which can be journal article, conference proceedings paper, book chapter, technical report, etc., and the country that the study was conducted. The method identification category includes criteria that deal with the assessment method developed or employed. Therefore, the criterion description of the assessment tool describes the method or tool presented based on whether it is a presentation of a new methodology, the application of an existing method or tool or a combination namely a new methodology that is implemented with an application example. The last criterion is the level of the assessment performed, i.e., global, national, regional, or farm level, according to the approach introduced by Gomez-Limon et al. (2010) [24]. After the initial assessment and finalization, for the sample to be reviewed, phase is completed; the in-depth review stage follows. For this stage, three (3) categories of criteria have been defined.
The first category of criteria assesses the scope of the studies reviewed. The first criterion is the identification of the goal (or objective) of each assessment, so as it is feasible to perform comparative reviews among studies with the same objective. For that purpose, following the definition of Gaviglio et al. (2017), the papers are classified according to whether a method is “goal prescribing or “system describing” [25]. Other criteria proposed concern the determination of the target user, as well as the functional unit and the time dimension of the assessment.
The second category refers to the identification of impacts starting with the definition of the sustainability dimension examined in each study, continuing with the documentation of the impacts considered during the assessment expressed in indicators (name and type). The last category concerns the data and the calculation methods used for the assessment. The criterion type of data examines whether the data used are model or experimental. Furthermore, to examine the accessibility of data, the present study refers to the definition of Angevin et al. (2017) [26]. Therefore, depending on the data used, the assessment can be characterized as ex ante (indicating expectation and uncertainty) when focusing on assessing a new scenario or as ex post (indicating processing actual field data) when examining a current situation [26]. Additionally, for each study reviewed, the validation and aggregation methods should be examined too.
The proposed methodological framework aims at facilitating the comparison among studies in order to capture the research advancements and current practices in the field under examination. This is an issue of particular importance since the assessment of agricultural sustainability is not a standardized process and entails a plethora of different methods, tools, and frameworks that assess a large number of different indicators that represent an analogously large number of different impacts. Prior to designing any assessment model, an exhaustive review is mandatory to safeguard consistency and relevance with other works. Also, the systematic documentation of the advancements in the field is the only way to begin constructing a unified, commonly accepted methodology for agricultural sustainability assessment.

3. Crop Production Sustainability at Farm Level

3.1. Search Scheme

The methodology presented above was used to investigate the available and mostly used methodologies to assess the sustainability of crop cultivations at the farm level. The review begins with the collection of the initial sample of papers by searching within the most acknowledged databases and more specifically, Scopus and Science Direct. The search scheme is based on specific keywords and their combination as presented in Table 2, and the use of Boolean operators (OR and AND) to increase the efficiency of the search. The initial search resulted in 959 papers containing the keywords searched. The initial sample was then screened based on the inclusion/exclusion criteria of Table 2. This secondary assessment resulted in 387 papers which where, then reviewed against the initial screening criteria (Figure 6).
As the purpose of this review is to examine studies assessing crop agricultural sustainability at the farm level, the 387-paper sample was filtered to select the peer-reviewed journal articles that fulfilled the following criteria. (a) Examine all three pillars of sustainability (environmental, economic, and social). (b) Examine production at the farm level. (c) Examine only crop cultivation. The final resulting collection of journal articles consists of 38 papers as presented in Table 3 and Appendix A.

3.2. Results

3.2.1. Initial Screening

As presented in the previous section 387 papers were reviewed in the initial screening stage. The filtering of the reviewed sample according to the scope of the review under study, resulted in 38 peer-reviewed journal articles. This section presents the initial systematic review of the 38-paper sample with the use of descriptive statistics to gain further insight about the general information that derive from the reviewed sample. With respect to the general information, the majority of papers (21%) were issued in 2017, whereas only two papers (5%) fitting the review criteria ware published in 2012, 2011, and 2010 [27]. However, it is worth noting that 45% of the examined papers was issued during the last three years (2016–2018), indicating a boost in the scientific community’s interest regarding integrated sustainability assessment (Figure 7).
Regarding the geographical origination, as presented in Figure 7, half of the assessments were performed in Europe (50%), whereas 16% were performed in Asia. Additionally, only three out of 38 assessments were performed in North America. With respect to the literature typology of the studies reviewed, as it was mentioned before only peer-reviewed journal articles were included in the reviewed sample.
Regarding the method identification category, Table 3 presents all the methods and tools that were identified during the review process (the nomenclature is presented in Appendix A). All of the relevant methods will be presented in detail later. In the majority of the papers examined (66%), the methods or tools presented are also practically tested presenting the relevant examples (case studies). In 18% of the papers, an already existing methodology was applied and presented while 16% of papers presented a methodology without testing it in practice. Continuing with the level of assessment, in 79% of the works examined, the assessment was performed exclusively for the farm level, whereas for 21% of the works, the level of assessment was also broadened beyond the farm level by examining local, regional, or national sustainability. The most frequently examined crop is maize and wheat (examined in five cases each), followed by olive, spinach and rice (examined in two cases studies each). The other crops examined in the papers reviewed included legumes, lettuce, scallions, red radish, banana, soybean, grapes, cranberry, potato, and coffee. Additionally, different agronomic practices are examined as for example organic farms [28], greenhouse cultivations [29], and school gardens [30].

3.2.2. In-Depth Review

This section presents the systematic review results against the in-depth review criteria initializing the presentation with the scope criteria category (Table A2 and Table A3 of Appendix A). Regarding the goal of the assessment, 61% of the examined studies are system describing, whereas the other 40% attempts to identify and evaluate policies and techniques that could be used to improve agricultural sustainability performance. Regarding the target users of the methodologies proposed, the majority of the examined works is aimed at decision-makers, farmers, and researchers. More specifically, 40% of the studies identify decision-makers as their target users, whereas 26% aim at farmers and 21% aim at researchers. Continuing, only three (3) works define a functional unit as a basis for the assessment. In particular, De Luca et al. (2018), when examining the sustainability of olive growing systems, proposed “One hectare (1ha) of cultivated surface” as a functional unit [31]. On the other hand, Theurl et al. (2017) and El Chami et al. (2015) preferred functional units related to the weight of the final product (“kg of un-/packed fresh product at the point of sale-POS” and “1 tn fresh weight standardized to 86% dry matter, respectively”) [36,43].
Concerning the criterion of the time dimension, in several studies the assessment was performed for a single year period [25,28,29,33,36,41,51,60]. However, there are also studies that perform the assessment for a range of years. Snapp et al. (2018) performed a 3-year trial, and Vasileiadis et al. (2017) extracted their data during a 4-year experiment [32,38]. Sharma et al. (2011) collected data from three separate decades from 1950 to present [55], and Gomes et al. (2009) collected data from 1986 to 2002 [57]. From another point of view, De Luca et al. (2018) expanded their assessment to the life cycle of an olive tree orchard (50 years) [31], and el Chami et al. (2015) projected the assessment to 2050 [43].
Regarding the Impact Identification category, as described above, the research scope contains only studies that attempt to examine all the three dimensions of sustainability, namely, the environmental, economic, as well as social pillar, contributing towards an integrated sustainability assessment evaluation. During the extensive review, all of the individual impacts—expressed as indicators—that were examined within the reviewed studies were extracted and documented. However, further thorough classification and commenting on the individual indicators used goes beyond the limits of this analysis and has already been investigated in several review studies in the past [4,13,18,19,22].
With respect to the data calculation method category of criteria, for 82% of the papers examined a validation process is not mentioned. Only 18% of the papers describe a validation process for the proposed methodologies. On the other hand, 74% of the studies mention the use of an aggregation technique or methodology aiming at the simplification and the generalization of the results. Regarding the type of data used for the assessments performed (Figure 8), the majority uses experimental data (68%), whereas a small percentage of works (18.4%) employ only model data for the sustainability assessment. Accordingly, 58% are ex post assessments attempting to evaluate current practices; whereas, in 31.6% of the papers, the evaluation of prediction scenarios is attempted.

3.3. Agricultural Sustainability Methods and Tools

In the previous sections a descriptive qualitative analysis of the review criteria was presented. The aim was to examine the research trend of crop agricultural sustainability and specifically the trend of the criteria concerning the scope and the calculation methods used. In this section, the methodologies and tools, extracted as a result of the review conducted, are presented. Figure 9 demonstrates the methods and tools identified and the corresponding frequency of occurrence. These methods and tools were classified in five major categories based on the main scope of the assessment (as expressed by the authors), underlining the fact that the categories selected may overlap as part of the overall concept. A distinctive example is MCDA which is used to facilitate the assessment of multivariate problems that are expressed with indicators. Nevertheless, the scope of studies employing MCDA methods focus on the aggregation of the results while methods proposing indicator sets and indexes focus on determining the criteria of the assessment. Another example is the carbon footprint (CF) which is an indicator that is often met in Indicators sets and frameworks. Nevertheless, it is a very commonly used standalone methodology for environmental impact assessment.
To that end, LCA methods relate to the life cycle of the examined element. Environmental methods relate to the quantification of the environmental impact of the examined element, and economic methods refer to the use of financial methods in the impact assessment. Multicriteria methods are methods that employ multicriteria assessment for the evaluation of agricultural sustainability, and Indicator methods include indicator sets and frameworks for the assessment of agricultural sustainability. With respect to the individual methodologies that were identified, the term “indicators” refers to all those methodologies that were not given a specific name by their developers.

3.3.1. Life Cycle Assessment, Environmental, and Economic Methods and Tools

For 21% of the studies reviewed, methods belonging to LCA, environmental, or economic method categories are employed. El Chami et al. (2015) performed an integrated sustainability assessment comparing different irrigation scenarios of winter wheat production at the farm level by proposing a methodology that combined LCA, SCC, and CBA [43]. Following the concept of LCA, Theurl et al. (2017) assessed the environmental and socio-economic impacts of unheated soil-grown vegetables [36]. Theurl et al. performed a comparative assessment utilizing experimental field data and data collected from literature, calculating the GHG emissions with socio-economic indicators deriving from the Sustainability Assessment of Food and Agriculture Systems (SAFA) guidelines of the Food and Agriculture Organization (FAO) [36]. Theurl et al. combined methodologies from three of the five categories identified, namely the LCA, the environmental and the indicator methods.
From the most recent studies, De Luca et al. (2018) assess the sustainability of olive growing systems by focusing on scenarios differentiated in weeding [31]. For the assessment, authors combined a series of tools to evaluate the three pillars of sustainability, namely, LCA for the environmental pillar, LCC for the economic, and SLCA for the societal pillar. They integrated their results by employing the AHP method for multicriteria analysis [31]. From the economic methods category, Van Passel et al. (2009) proposed a methodological framework based on the sustainable value approach (SVA) to assess the sustainability on farm production level [58]. Van Passel et al. employed the SVA method attempting to correlate farm performance in respect to consumption of resources. The work represents a benchmarking approach since it does not focus on the evaluation of sustainability in absolute terms, but it assesses the performance compared to standards [58]. Van Passel et al. (2011) stated that to perform multilevel and multi-user assessments, a combination of methodologies can offer more advantages than integrated methodologies [53]. To that end, the SVA method was combined with the MOTIFS indicator tool. According to Van Passel et al. (2011), MOTIFS is a visual monitoring tool used for the aggregation of indicators of various themes, which creates benchmarks for the rescaling of the indicator values [53].

3.3.2. Multicriteria Assessment Methods and Tools

Within the multicriteria assessment methods that are used for assessing agricultural sustainability, the works examined can be classified into groups that employ and develop the same methodological framework. Such groups are the studies that use the MASC decision model developed by Sadok et al. (2009), which was built as part of the decision support system DEXi [59]. The MASC model is a hierarchical multiattribute decision support model designed for the ex ante assessment of cropping systems to address the need of in-field alternative scenario evaluation. Such models allow for the simplification of the decision problem by downscaling it to smaller and less complex problems expressed by designated variables [59]. The DEX methodology performs aggregation of qualitative attributes and utility functions using “IF-THEN” aggregation rules [59]. Colomb et al. (2013) building upon Sadok’s et al. (2009) model, proposed the MASC-OF model to assess the strong and weak points of organic cropping systems in a regional context [50]. Pelzer et al. (2012) also following Sadok et al. (2009) presented the DEXiPM model which was particularly developed for integrated pest management systems (IPM). The DEXiPM model is built upon the MASC model and it is an ex ante methodology contributing towards the discussion around innovative systems. The model was implemented in winter crop and maize-based cropping systems and consists of seventy-five (75) basic and eighty-six (86) aggregated indicators [27].
Vasileiadis et al. (2013) used the DEXiPM model to compare the sustainability of innovative IPM-based systems [51]. Also, Vasileiadis et al. (2017) used Pelzer’s et al. (2012) DEXiPM model, which was supplemented by Angevin et al. (2017), for the ex post assessment of the economic, environmental and social sustainability of conventional winter wheat and maize cropping systems [26]. The IPM-based systems were designed and tested in nine (9) locations in Europe [38]. They compared the sustainability of the examined systems, discussing the benefits or drawbacks of the IPM systems. Vasileiadis et al. (2017) also adopted methodologies from the environmental and economic categories. Economic data, with the use of a template, were collected from participants to perform cost–benefit analysis (CBA). Furthermore, an environmental risk assessment was performed by implementing the SYNOPS-WEB Tool [38]. Lastly, Chopin et al. (2017) adapted the MASC model in order to ex ante assess the sustainability in the area of local banana farming systems [37].
Multicriteria methods facilitate decision making while considering multiple variables, and such methods use weighting techniques in order to produce composite indices [24]. Among the studies examined, the most frequently used methods are the principal component analysis (PCA) and the data envelopment analysis (DEA). Specifically, Gomez-Limon et al. (2010) and Sottile et al. (2016) used the PCA method, whereas Gomes et al. (2009) and Reig-Martinez et al. (2011) used the DEA method to create a composite indicator [24,30,54,57]. Dong et al. (2015 and 2016) combined both methodologies attempting to construct a complex indicator composed of a large number of individual and interdepended variable [40,46].
Concluding with the multicriteria method category, Siciliano et al. (2009) used the social multicriteria evaluation (SMCE) framework, which was implemented through the NAIADE (novel approach to imprecise assessment and decision environments) software, to assess the sustainability of farming practices in a small rural area in Italy [60]. Egea et al. (2016) employed the analytic hierarchy process (AHP) in order to investigate the combination of protected destination of origin oil production system that leads to optimal sustainability [39]. Bockstaller et al. (2017) introduced the CONTRA tool, an innovative aggregation method that leads to the creation of decision trees using fuzzy sets [34]. Peano et al. (2014) proposed a multicriteria methodology to evaluate the effectiveness of the slow food presidia, which are organized structures aiming at the preservation of quality production at risk to extinction by following specific guidelines and protocols for each product category [48].

3.3.3. Indicator Sets, Indexed and Frameworks

This category of methods and tools contains indicator sets, indexes, and frameworks that were used in the reviewed works to assess agricultural sustainability at the farm level. Walter et al. (2009a; 2009b) proposes a new indicator-based method to assess the unsustainability of a system rather than its sustainability [62]. Their method borrows elements of the LCA methodology and was implemented in two stages. The first stage includes the creation of an issue inventory and its contextualization, while the second stage includes the standardization and sustainability valuation process [61,62]. Rodriguez et al. (2010) proposed the APOIA-NovoRural framework, which comprises a collection of basic and composite indicators covering five dimensions of sustainability: landscape ecology, environmental quality, sociocultural values, economic values, and management and administration [56]. Sharma et al. (2011) introduced a methodology based on questionnaires and surveys and composed an agricultural sustainability index (ASI) targeted to Bihar province (India) [55], and also calculated the sustainability parameters for a 60-year period.
Sami et al. (2013) selected six indicators that were considered appropriate to assess sustainability in a regional context. Additionally, in order to evaluate some of these indicators they used a selection of fuzzy submodels [52]. Van Asselt et al. (2014) propose a protocol for the collection and evaluation of indicators for the sustainability assessment of agri-food production systems [49]. Their proposed list covers a wide range of indicators related to the three pillars of sustainability, aiming at supporting policy makers in decision making by choosing the most relevant indicators. Yegbemey et al. (2014), proposed an innovative participatory approach that resulted in seventeen (17) indicators. All relevant data were collected through a household survey. The sustainability was evaluated with relative scores while the total sustainability level was based on the average scores of the individual indicators [47]. Peano et al. (2015), proposed the SAEMETH monitoring tool based on a set of qualitative indicators. The selection of the indicators was based on the criteria introduced by Meul et al. (2008) [63] and, for their evaluation, they set a minimum and maximum threshold based on reference values that was derived from best practices or through surveys [45]. Santiago-Brown et al. (2015) presented the process for selecting indicators to assess viticulture production sustainability. For the selection of the indicators, the adapted nominal group technique was used. The selected indicators were reduced according to their relevance [44] resulting in seventy-six (76) indicators hierarchized based on their importance.
Allahyari et al. (2016) selected five-hundred-and-eighty-eight (588) indicators through an extensive literature review. Following erasing duplicates and prioritizing the sample, it resulted in 62 indicators, which were used in an extensive survey among experts. The indicators were assessed based on their importance while the resulting data were assessed with the Minskowski fuzzy screening method [42]. Sajjad et al. (2016) examined the relevant agricultural sustainability at farm and regional scale using the sustainable livelihood security index (SLSI) [41]. Yang et al. (2016) assessed the sustainability of greenhouse vegetables using indicators. More specifically, to examine the greenhouse vegetable farming practices and the economic and social management conditions, they used rapid and participatory rural appraisal (RRA/PRA) tools combined with data derived from in-field measurements and parallel surveys [29]. In 2016, de Olde et al. proposed the sustainability assessment tool named response-inducing sustainability evaluation (RISE), which was implemented for the evaluation of organic farms in Denmark. The tool contains indicators for a total of 10 themes and 51 subthemes. The indicators were normalized and aggregated and each theme was evaluated based on the average score of the relevant subthemes [28].
Goswami et al. (2017) integrated the sustainable livelihood (SL) and the drivers–pressures–state–impact–response (DPSIR) framework, proposing a small farm sustainability index (SFSI) that could address the complexity of small-holder family farms under a participatory approach [35]. The proposed framework assesses sustainability in multiple levels assigning the relevant weights and resulting in the creation of an aggregated index for the entire system. They indicate that the introduction ICT technologies in agriculture (web-based platforms, wireless sensors, etc.) can facilitate data sharing among stakeholders and provide the basis for assessing the sustainability of farming systems. Recanati et al. (2017) proposed an indicator-based framework for the assessment of sustainability of small-scale farming systems in water-limited regions. They implemented the framework by modeling an “average” farm based on a survey among 30 farmers [33]. Gaviglio et al. (2017), attempting to integrate various analytical techniques, introduced the 4AGRO tool, which is an online self-assessment tool based on indicators. It consists of 42 subindicators that are divided in 15 complex indicators, five for each pillar of sustainability [25]. The tool was demonstrated in an agricultural park in Italy. Finally, Snapp et al. (2018) proposed a methodology based on indicators that derived through a participatory approach involving a steering committee with multidisciplinary participants from eight (8) institutions [32]. The indicators were normalized based on max possible values.

4. Conclusions

To meet the ever-increasing interest towards agricultural sustainability, many methodologies and tools emerge, introducing integrated and holistic assessment approaches. However, there is still no consensus on the standardization of agricultural sustainability assessment as part of a unified concept of sustainable development. Newly introduced frameworks propose mostly case-specific tools that focus on resource use and their impact on the sustainability of farming practices. Combinational use of methodologies is observed in many cases; thus, a clear distinction of methodologies is not always possible. Contributing towards the indexing of the available methodologies, the present paper presented a methodological framework for the systematic literature review of agricultural sustainability studies. The framework synthesizes all the available literature review criteria and introduces a two-level analysis facilitating systematization, data mining, and methodology extraction.
The framework was implemented for the systematic literature review of crop agricultural sustainability assessment studies at farm-level for the last decade. The investigation of the methodologies used is of particular importance since there are no standards or norms for the sustainability assessment of farming practices. The chronological analysis revealed that the scientific community’s interest in agricultural sustainability has been increasing during the last three (3) years, indicating a tendency to gradually progress from the theory of economic growth to the more comprehensive and inclusive concept of sustainable development. Nevertheless, the critical evaluation of effectiveness and the implications of the methods presented are outside the scope of the present work and are subjects of thorough future research.
The most used methods include indicator-based tools, frameworks and indexes followed by multicriteria methods. In the reviewed studies, stakeholder participation is proved crucial in the determination of the level of sustainability. However, a systematic assessment of the agricultural machinery’s and operation management’s contribution to the overall sustainability was not detected in the examined studies. The effect of resource use and input management is the most usually examined issue in the reviewed studies.

Author Contributions

Conceptualization, D.B. and M.G.L.; methodology, M.G.L.; validation, D.B. and C.G.S.; investigation, M.G.L.; data curation, C.G.S.; writing—original draft preparation, M.G.L.; writing—review and editing, D.B. and C.G.S.; visualization, M.G.L.; supervision, D.B.


The work was supported by the project “Research Synergy to address major challenges in the nexus: energy–environment–agricultural production (Food, Water, Materials)”—NEXUS, funded by the Greek Secretariat for Research and Technology (GSRT)—Pr. No. MIS 5002496.

Conflicts of Interest

The authors declare no conflicts of interest.


LCALife Cycle Assessment
LCCLife Cycle Costing
SLCASocial Life Cycle Assessment
SYNOPS WEBEnvironmental Risk Assessment Web tool
CFCarbon Footprint
SCCSocial Cost of Carbon
CBACost–benefit Analysis
SVASustainable Value Approach
MCDAMulti Criteria Decision Analysis
AHPAnalytic Hierarchy Process
ANPAnalytical Network Process
MASCMultiattribute Assessment of Cropping Systems
DEXiA Program for Multiattribute Decision Making
DEXiPMDEXi Pest Management
SMCESocial Multicriteria Evaluation
NAIADENovel approach to imprecise assessment and decision environments
PCAPrinciple Components Analysis
CONTRAthe French acronym for ‘design of transparent decision trees’
DEAData Envelopment Analysis
RISEResponse-Inducing Sustainability Evaluation
SLSISustainable Livelihood Security Index
SFSISmall Farm Sustainability Index
SAEMETHSustainable Agri-Food Evaluation Methodology
ANGT Adapted Nominal Group Technique
MOTIFSMonitoring Tool for Integrated Farm Sustainability
ASIAgricultural Sustainability Index
APOIA-NovoRUralA system for weighted environmental impact assessment of rural activities
SAFEA hierarchical framework for assessing the sustainability of agricultural systems
DPSIRDrivers, Pressures, State, Impact, Response

Appendix A

Table A1. Review criteria frequency table.
Table A1. Review criteria frequency table.
CriteriaDe Luca et al., 2017 [10]Binder et al., 2010 [4]Peter et al., 2017 [12]Bockstaller et al., 2009 [3]Roy et al., 2012 [13]Cerutti et al.,2011 [7]Acosta-Alba et al.,2011 [14]Baldini et al.,2017 [15]Morais et al., 2016 [16]McAuliffe et al., 2016 [17]Latruffe et al.,2016 [18]Bockstaller et al., 2008 [19]Payraudeau et al.,2005 [20]de Vries et al., 2015 [21]Yan et al., 2011 [9]Lebacq et al.,2013 [22]
1Sustainability Assessment method++++++++++ +++++
2Sustainability Assessment Tool+ + +
3Description of Assessment Tool +++ + +
4Field of Application/Product+ ++++ + +
5Year of Publication+ +++ +++ +++
6Country+ + ++ + ++
7Literature Typology+ + ++
8Typology of Study+ +
9Level of Assessment/System Boundaries++++++++ ++ ++
10Goal of the Assessment ++ +++ + ++ ++
11Views of the Assessment +
12Assessment Approach/Standards + +
13Database Used +
14User Interface method +
15Developer of the method ++
16Strengths+ +
17Drawbacks+ +
18Validation ++ + +
19Readability + +
20Feasibility + +
21Reproducibility + +
22Relevance for end users + +
23Target Users++ + + ++
24Aggregation+ + + +
25Rating Scores+ +
26Thresholds/Characterization factors + ++
27Type of Data used ++ + ++ +
28Accessibility of data +++ + + +
29Indicator Name/Type++++++ +++ +
30Integration with existing farming software +
31Need for external support +
32User-Friendliness +
33Transparency +
34Calculation method + + +
35Impacts Considered/Impact Identification+ ++++ +++ +++
36Agronomic Protocol + +
37Functional Unit Category +
38Operation with the greatest impact +
39Dimension of Sustainability Studied +++ + + + +
40Functional Unit+ + +++++ +++
41Uncertainty and Sensitivity Analysis + + +
42Case study Description ++
43Participation method++
44Time Dimension ++ + +
Table A2. Scope (38-paper collection).
Table A2. Scope (38-paper collection).
a/aAuthorAssessment GoalTarget UsersFunctional UnitTime Dimension
1De Luca et al. (2018) [31]SDRE TE FA1ha cultivated surface50 years
2Snapp et al. (2018) [32]GP -3 years mother trial
3Gaviglio et al. (2017) [25]SDDM FA RE-1 year
4Recanati et al. (2017) [33]GP -1 year
5Bockstaller et al. (2017) [34]GPDM FA RE--
6Goswami et al. (2017) [35]SD -
7Theurl et al. (2017) [36]GP kg un-/pacjed fresh producty at the POS10/2014-04/2015
8Chopin et al. (2017) [37]GPDM RE FA--
9Angevin et al. (2017) [26]GPDM RE--
10Vasileiadis et al. (2017) [38]GPDM FA-4-year experiment
11Egea et al. (2016) [39]GP --
12Dong et al. (2016) [40]SD --
13de Olde et al. (2016) [28]SDDM FA RE-2013-2014
14Yang et al. (2016) [29]SD -Fall 2011-spring 2012 (Sampling)
15Sajjad et al. (2016) [41]SD--2012-2013
16Allahyari et al. (2016) [42]SD---
17Sottile et al. (2016) [30]SD---
18El Chami et al. (2015) [43]GP-1tn fresh weight standardizes to 86% dry matterProjection to 2050
19Santiago-Brown et al. (2015) [44]SDDM FA RE--
20Peano et al. (2015) [45]SDFA ΤΕ-4-6 years in the ex post stage
21Dong et al. (2015) [46]SD---
22Yegbemey et al. (2014) [47]SD--
23Peano et al. (2014) [48]GP---
24Van Asselt et al. (2014) [49]GP -
25Colomb et al. (2013) [50]GPDM--
26Vasileiadis et al. (2013) [51]GP--2012
27Sami et al. (2013) [52]SD---
28Pelzer et al. (2012) [27]GPDM FA--
29Van Passel et al. (2012) [53]SD --
30Reig-Martinez et al. (2011) [54]SD---
31Sharma et al. (2011) [55]SD -3 separate decades from 1950
32Rodriguez et al. (2010) [56]SDDM RE FA TE-
33Gomez-Limon et al. (2010) [24]SDDM-
34Gomes et al. (2009) [57]SD--1986-2002
35Van Passel et al. (2009) [58]SDDM--
36Sadok et al. ([59]SDDM--
37Siciliano (2009) [60]GP -2003
38Walter et al. (2009) [61,62]SD -
SD: System describing; GP: Goal prescribing; DM: Decision-maker; RE: Researcher; TE: Technician; FA: Farmer.
Table A3. Data calculation method (38-paper collection).
Table A3. Data calculation method (38-paper collection).
a/aAuthorValidationAggregationType of DataAccessibility of Data
1De Luca et al. (2018) [31]- E (Survey)Ex ante
2Snapp et al. (2018) [32]--EEx post
3Gaviglio et al. (2017) [25]-SumΕ (Survey)Ex ante
4Recanati et al. (2017) [33]- M (Model Farm based on survey)Ex ante
5Bockstaller et al. (2017) [34]-Sum (rank of percentage of weight)E, M
6Goswami et al. (2017) [35]D, O, ULinear, Geometric, Multicriteria function-based-Ex ante
7Theurl et al. (2017) [36]- E (Data from field and survey)Ex post
8Chopin et al. (2017) [37]-Decision Rules and relative weightings (using direct scoring method)Μ, Ε (When model not available)Ex ante
9Angevin et al. (2017) [26]EU (Experts Evaluation)“IF-THEN” Aggregation rulesEex post
10Vasileiadis et al. (2017) [38]-“IF-THEN” Aggregation rulesΕex post
11Egea et al. (2016) [39] YES (weighted sum)E (Expert opinion)Ex post
12Dong et al. (2016) [40]DPCA and DEA (Weighting Factors)E (Survey)Ex post
13de Olde et al. (2016) [28]-Arithmetic meanEEx post
14Yang et al. (2016) [29]--E (Survey)Ex post
15Sajjad et al. (2016) [41]-Not specificE (Survey)Ex post
16Allahyari et al. (2016) [42]-Mean methodE (Survey to assess indicators)Ex post
17Sottile et al. (2016) [30]-PCA, Cluster AnalysisE (survey)Ex post
18El Chami et al. (2015) [43]--MEx ante
19Santiago-Brown et al. (2015) [44]----
20Peano et al. (2015) [45]ΟBasic Indicators as equally importantE (Semi-structured interviews)Ex ante and ex post
21Dong et al. (2015) [46]-DEA modelE (Survey)Ex post
22Yegbemey et al. (2014) [47] Simple and Linear aggregation techniqueE(Survey)Ex post
23Peano et al. (2014) [48]DEqual weights or based on importanceEEx post
24Van Asselt et al. (2014) [49]-Normalization between 0-100 then weights based on importanceEEx post
25Colomb et al. (2013) [50]NO“IF-THEN” Aggregation rulesEEx ante
26Vasileiadis et al. (2013) [51]-“IF-THEN” Aggregation rulesE and MEx ante
27Sami et al. (2013) [52]-“IF-THEN” Aggregation rulesΕEx post
28Pelzer et al. (2012) [27]U“IF-THEN” Aggregation rulesMEx ante
29Van Passel et al. (2012) [53]YESThe indicators are integrated into MOTIFS graphM (Farm Accountancy data)Ex post
30Reig-Martinez et al. (2011) [54]-DEA modelΕ (Survey)Ex post
31Sharma et al. (2011) [55] Equal weightsΕ (Survey)Ex post
32Rodriguez et al. (2010) [56]-Normalization and weights then average for indicesEEx post
33Gomez-Limon et al. (2010) [24]-Weighted sum, Product of weighted indicatorsE (Survey)Ex post
34Gomes et al. (2009) [57]--EEx post
35Van Passel et al. (2009) [58]--M (Empirical data)Ex ante
36Sadok et al. ([59]-“IF-THEN” Aggregation rulesMEx ante
37Siciliano (2009) [60]-Not specificEEx post and ex ante
38Walter et al. (2009) [61,62] Sum and Weighted sumMEx ante
M: Model, E: Experimental.


  1. United Nations Department of Economic and Social Affairs. World Population Projection; United Nations Department of Economic and Social Affairs: New York, NY, USA, 2017. [Google Scholar]
  2. Viola, I.; Marinelli, A. Life Cycle Assessment and Environmental Sustainability in the Food System. Agric. Agric. Sci. Procedia 2016, 8, 317–323. [Google Scholar] [CrossRef] [Green Version]
  3. Bockstaller, C.; Guichard, L.; Keichinger, O.; Girardin, P.; Galan, M.B.; Gaillard, G. Comparison of methods to assess the sustainability of agricultural systems. A review. Agron. Sustain. Dev. 2009, 29, 223–235. [Google Scholar] [CrossRef] [Green Version]
  4. Binder, C.R.; Feola, G.; Steinberger, J.K. Considering the normative, systemic and procedural dimensions in indicator-based sustainability assessments in agriculture. Environ. Impact Assess. Rev. 2010, 30, 71–81. [Google Scholar] [CrossRef]
  5. De Olde, E.M.; Moller, H.; Marchand, F.; McDowell, R.W.; MacLeod, C.J.; Sautier, M.; Halloy, S.; Barber, A.; Benge, J.; Bockstaller, C.; et al. When experts disagree: The need to rethink indicator selection for assessing sustainability of agriculture. Environ. Dev. Sustain. 2017, 19, 1327–1342. [Google Scholar] [CrossRef]
  6. Van Pham, L.; Smith, C. Drivers of agricultural sustainability in developing countries: A review. Environ. Syst. Decis. 2014, 34, 326–341. [Google Scholar] [CrossRef]
  7. Cerutti, A.K.; Bruun, S.; Beccaro, G.L.; Bounous, G. A review of studies applying environmental impact assessment methods on fruit production systems. J. Environ. Manag. 2011, 92, 2277–2286. [Google Scholar] [CrossRef] [PubMed]
  8. De Olde, E.M.; Oudshoorn, F.W.; Sørensen, C.A.G.; Bokkers, E.A.M.; De Boer, I.J.M. Assessing sustainability at farm-level: Lessons learned from a comparison of tools in practice. Ecol. Indic. 2016, 66, 391–404. [Google Scholar] [CrossRef]
  9. Yan, M.J.; Humphreys, J.; Holden, N.M. An evaluation of life cycle assessment of European milk production. J. Environ. Manag. 2011, 92, 372–379. [Google Scholar] [CrossRef]
  10. De Luca, A.I.; Iofrida, N.; Leskinen, P.; Stillitano, T.; Falcone, G.; Strano, A.; Gulisano, G. Life cycle tools combined with multi-criteria and participatory methods for agricultural sustainability: Insights from a systematic and critical review. Sci. Total Environ. 2017, 595, 352–370. [Google Scholar] [CrossRef]
  11. Grant, M.J.; Booth, A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Inf. Libr. J. 2009, 26, 91–108. [Google Scholar] [CrossRef]
  12. Peter, C.; Helming, K.; Nendel, C. Do greenhouse gas emission calculations from energy crop cultivation reflect actual agricultural management practices?—A review of carbon footprint calculators. Renew. Sustain. Energy Rev. 2017, 67, 461–476. [Google Scholar] [CrossRef]
  13. Roy, R.; Chan, N.W. An assessment of agricultural sustainability indicators in Bangladesh: Review and synthesis. Environmentalist 2012, 32, 99–110. [Google Scholar] [CrossRef]
  14. Acosta-Alba, I.; van der Werf, H.M.G. The use of reference values in indicator-based methods for the environmental assessment of agricultural systems. Sustainability 2011, 3, 424–442. [Google Scholar] [CrossRef]
  15. Baldini, C.; Gardoni, D.; Guarino, M. A critical review of the recent evolution of Life Cycle Assessment applied to milk production. J. Clean. Prod. 2017, 140, 421–435. [Google Scholar] [CrossRef]
  16. Morais, T.G.; Teixeira, R.F.M.; Domingos, T. Regionalization of agri-food life cycle assessment: A review of studies in Portugal and recommendations for the future. Int. J. Life Cycle Assess. 2016, 21, 875–884. [Google Scholar] [CrossRef]
  17. McAuliffe, G.A.; Chapman, D.V.; Sage, C.L. A thematic review of life cycle assessment (LCA) applied to pig production. Environ. Impact Assess. Rev. 2016, 56, 12–22. [Google Scholar] [CrossRef]
  18. Latruffe, L.; Diazabakana, A.; Bockstaller, C.; Desjeux, Y.; Finn, J.; Kelly, E.; Ryan, M.; Uthes, S. Measurement of sustainability in agriculture: A review of indicators. Stud. Agric. Econ. 2016, 118, 123–130. [Google Scholar] [CrossRef]
  19. Bockstaller, C.; Guichard, L.; Makowski, D.; Aveline, A.; Girardins, P.; Plantureux, S. Review article Agri-environmental indicators to assess cropping and farming systems. A review. Agron. Sustain. Dev. 2008, 28, 139–149. [Google Scholar] [CrossRef]
  20. Payraudeau, S.; Van Der Werf, H.M.G. Environmental impact assessment for a farming region: A review of methods. Agric. Ecosyst. Environ. 2005, 107, 1–19. [Google Scholar] [CrossRef]
  21. De Vries, M.; van Middelaar, C.E.; de Boer, I.J.M. Comparing environmental impacts of beef production systems: A review of life cycle assessments. Livest. Sci. 2015, 178, 279–288. [Google Scholar] [CrossRef]
  22. Lebacq, T.; Baret, P.V.; Stilmant, D. Sustainability indicators for livestock farming. A review. Agron. Sustain. Dev. 2013, 33, 311–327. [Google Scholar] [CrossRef]
  23. Bockstaller, C.; Guichard, L.; Keichinger, O.; Girardin, P.; Galan, M.B.; Gaillard, G. Comparison of methods to assess the sustainability of agricultural systems: A review. In Sustainable Agriculture; Springer: Berlin/Heidelberg, Germany, 2009; ISBN 9789048126651. [Google Scholar]
  24. Gómez-Limón, J.A.; Sanchez-Fernandez, G. Empirical evaluation of agricultural sustainability using composite indicators. Ecol. Econ. 2010, 69, 1062–1075. [Google Scholar] [CrossRef]
  25. Gaviglio, A.; Bertocchi, M.; Demartini, E. A Tool for the Sustainability Assessment of Farms: Selection, Adaptation and Use of Indicators for an Italian Case Study. Resources 2017, 6, 60. [Google Scholar] [CrossRef]
  26. Angevin, F.; Fortino, G.; Bockstaller, C.; Pelzer, E.; Messéan, A. Assessing the sustainability of crop production systems: Toward a common framework? Crop Prot. 2017, 97, 18–27. [Google Scholar] [CrossRef]
  27. Pelzer, E.; Fortino, G.; Bockstaller, C.; Angevin, F.; Lamine, C.; Moonen, C.; Vasileiadis, V.; Guérin, D.; Guichard, L.; Reau, R.; et al. Assessing innovative cropping systems with DEXiPM, a qualitative multi-criteria assessment tool derived from DEXi. Ecol. Indic. 2012, 18, 171–182. [Google Scholar] [CrossRef]
  28. De Olde, E.; Oudshoorn, F.; Bokkers, E.; Stubsgaard, A.; Sørensen, C.; de Boer, I. Assessing the Sustainability Performance of Organic Farms in Denmark. Sustainability 2016, 8, 957. [Google Scholar] [CrossRef]
  29. Yang, L.; Huang, B.; Mao, M.; Yao, L.; Niedermann, S.; Hu, W.; Chen, Y. Sustainability assessment of greenhouse vegetable farming practices from environmental, economic, and socio-institutional perspectives in China. Environ. Sci. Pollut. Res. 2016, 23, 17287–17297. [Google Scholar] [CrossRef] [PubMed]
  30. Sottile, F.; Fiorito, D.; Tecco, N.; Girgenti, V.; Peano, C. An interpretive framework for assessing and monitoring the sustainability of school gardens. Sustainability 2016, 8, 801. [Google Scholar] [CrossRef]
  31. De Luca, A.I.; Falcone, G.; Stillitano, T.; Iofrida, N.; Strano, A.; Gulisano, G. Evaluation of sustainable innovations in olive growing systems: A Life Cycle Sustainability Assessment case study in southern Italy. J. Clean. Prod. 2018, 171, 1187–1202. [Google Scholar] [CrossRef]
  32. Snapp, S.S.; Grabowski, P.; Chikowo, R.; Smith, A.; Anders, E.; Sirrine, D.; Chimonyo, V.; Bekunda, M. Maize yield and profitability tradeoffs with social, human and environmental performance: Is sustainable intensification feasible? Agric. Syst. 2018, 162, 77–88. [Google Scholar] [CrossRef]
  33. Recanati, F.; Castelletti, A.; Dotelli, G.; Melià, P. Trading off natural resources and rural livelihoods. A framework for sustainability assessment of small-scale food production in water-limited regions. Adv. Water Resour. 2017, 110, 484–493. [Google Scholar] [CrossRef] [Green Version]
  34. Bockstaller, C.; Beauchet, S.; Manneville, V.; Amiaud, B.; Botreau, R. A tool to design fuzzy decision trees for sustainability assessment. Environ. Model. Softw. 2017, 97, 130–144. [Google Scholar] [CrossRef]
  35. Goswami, R.; Saha, S.; Dasgupta, P. Sustainability assessment of smallholder farms in developing countries. Agroecol. Sustain. Food Syst. 2017, 41, 546–569. [Google Scholar] [CrossRef]
  36. Theurl, M.C.; Hörtenhuber, S.J.; Lindenthal, T.; Palme, W. Unheated soil-grown winter vegetables in Austria: Greenhouse gas emissions and socio-economic factors of diffusion potential. J. Clean. Prod. 2017, 151, 134–144. [Google Scholar] [CrossRef] [Green Version]
  37. Chopin, P.; Tirolien, J.; Blazy, J.M. Ex-ante sustainability assessment of cleaner banana production systems. J. Clean. Prod. 2016, 139, 15–24. [Google Scholar] [CrossRef]
  38. Vasileiadis, V.P.; Dachbrodt-Saaydeh, S.; Kudsk, P.; Colnenne-David, C.; Leprince, F.; Holb, I.J.; Kierzek, R.; Furlan, L.; Loddo, D.; Melander, B.; et al. Sustainability of European winter wheat- and maize-based cropping systems: Economic, environmental and social ex-post assessment of conventional and IPM-based systems. Crop Prot. 2017, 97, 60–69. [Google Scholar] [CrossRef]
  39. Egea, P.; Pérez y Pérez, L. Sustainability and multifunctionality of protected designations of origin of olive oil in Spain. Land Use Policy 2016, 58, 264–275. [Google Scholar] [CrossRef]
  40. Dong, F.; Mitchell, P.D.; Knuteson, D.; Wyman, J.; Bussan, A.J.; Conley, S. Assessing sustainability and improvements in US Midwestern soybean production systems using a PCA-DEA approach. Renew. Agric. Food Syst. 2016, 31, 524–539. [Google Scholar] [CrossRef]
  41. Sajjad, H.; Nasreen, I. Assessing farm-level agricultural sustainability using site-specific indicators and sustainable livelihood security index: Evidence from Vaishali district, India. Community Dev. 2016, 47, 602–619. [Google Scholar] [CrossRef]
  42. Allahyari, M.S.; Daghighi Masouleh, Z.; Koundinya, V. Implementing Minkowski fuzzy screening, entropy, and aggregation methods for selecting agricultural sustainability indicators. Agroecol. Sustain. Food Syst. 2016, 40, 277–294. [Google Scholar] [CrossRef]
  43. El Chami, D.; Daccache, A. Assessing sustainability of winter wheat production under climate change scenarios in a humid climate—An integrated modelling framework. Agric. Syst. 2015, 140, 19–25. [Google Scholar] [CrossRef]
  44. Santiago-Brown, I.; Metcalfe, A.; Jerram, C.; Collins, C. Sustainability assessment in wine-grape growing in the New World: Economic, environmental, and social indicators for agricultural businesses. Sustainability 2015, 7, 8178–8204. [Google Scholar] [CrossRef]
  45. Peano, C.; Tecco, N.; Dansero, E.; Girgenti, V.; Sottile, F. Evaluating the sustainability in complex agri-food systems: The SAEMETH framework. Sustainability 2015, 7, 6721–6741. [Google Scholar] [CrossRef]
  46. Dong, F.; Mitchell, P.D.; Colquhoun, J. Measuring farm sustainability using data envelope analysis with principal components: The case of Wisconsin cranberry. J. Environ. Manag. 2015, 147, 175–183. [Google Scholar] [CrossRef] [PubMed]
  47. Yegbemey, R.N.; Yabi, J.A.; Dossa, C.S.G.; Bauer, S. Novel participatory indicators of sustainability reveal weaknesses of maize cropping in Benin. Agron. Sustain. Dev. 2014, 34, 909–920. [Google Scholar] [CrossRef]
  48. Peano, C.; Migliorini, P.; Sottile, F. A methodology for the sustainability assessment of agri-food systems: An application to the slow food presidia project. Ecol. Soc. 2014, 19, 1–11. [Google Scholar] [CrossRef]
  49. Van Asselt, E.D.; Van Bussel, L.G.J.; Van Der Voet, H.; Van Der Heijden, G.W.A.M.; Tromp, S.O.; Rijgersberg, H.; Van Evert, F.; Van Wagenberg, C.P.A.; Van Der Fels-Klerx, H.J. A protocol for evaluating the sustainability of agri-food production systems—A case study on potato production in peri-urban agriculture in the Netherlands. Ecol. Indic. 2014, 43, 315–321. [Google Scholar] [CrossRef]
  50. Colomb, B.; Carof, M.; Aveline, A.; Bergez, J.E. Stockless organic farming: Strengths and weaknesses evidenced by a multicriteria sustainability assessment model. Agron. Sustain. Dev. 2013, 33, 593–608. [Google Scholar] [CrossRef]
  51. Vasileiadis, V.P.; Moonen, A.C.; Sattin, M.; Otto, S.; Pons, X.; Kudsk, P.; Veres, A.; Dorner, Z.; van der Weide, R.; Marraccini, E.; et al. Sustainability of European maize-based cropping systems: Economic, environmental and social assessment of current and proposed innovative IPM-based systems. Eur. J. Agron. 2013, 48, 1–11. [Google Scholar] [CrossRef]
  52. Sami, M.; Shiekhdavoodi, M.J.; Almassi, M.; Marzban, A. Assessing the sustainability of agricultural production systems using fuzzy logic. J. Cent. Eur. Agric. 2013, 14, 318–330. [Google Scholar] [CrossRef]
  53. Van Passel, S.; Meul, M. Multilevel and multi-user sustainability assessment of farming systems. Environ. Impact Assess. Rev. 2012, 32, 170–180. [Google Scholar] [CrossRef]
  54. Reig-Martínez, E.; Gómez-Limón, J.A.; Picazo-Tadeo, A.J. Ranking farms with a composite indicator of sustainability. Agric. Econ. 2011, 42, 561–575. [Google Scholar] [CrossRef]
  55. Sharma, D.; Shardendu, S. Assessing farm-level agricultural sustainability over a 60-year period in rural eastern India. Environmentalist 2011, 31, 325–337. [Google Scholar] [CrossRef]
  56. Rodrigues, G.S.; Rodrigues, I.A.; de Almeida Buschinelli, C.C.; de Barros, I. Integrated farm sustainability assessment for the environmental management of rural activities. Environ. Impact Assess. Rev. 2010, 30, 229–239. [Google Scholar] [CrossRef]
  57. Gomes, E.G.; Soares De Mello, J.C.C.B.; Souza, G.D.S.E.; Angulo Meza, L.; Mangabeira, J.A.D.C. Efficiency and sustainability assessment for a group of farmers in the Brazilian Amazon. Ann. Oper. Res. 2009, 169, 167–181. [Google Scholar] [CrossRef]
  58. Van Passel, S.; Van Huylenbroeck, G.; Lauwers, L.; Mathijs, E. Sustainable value assessment of farms using frontier efficiency benchmarks. J. Environ. Manag. 2009, 90, 3057–3069. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  59. Sadok, W.; Angevin, F.; Bergez, J.-E.; Bockstaller, C.; Colomb, B.; Guichard, L.; Reau, R.; Messéan, A.; Doré, T. MASC, a qualitative multi-attribute decision model for ex ante assessment of the sustainability of cropping systems. Agron. Sustain. Dev. 2009, 29, 447–461. [Google Scholar] [CrossRef] [Green Version]
  60. Siciliano, G. Social multicriteria evaluation of farming practices in the presence of soil degradation. A case study in Southern Tuscany, Italy. Environ. Dev. Sustain. 2009, 11, 1107–1133. [Google Scholar] [CrossRef]
  61. Walter, C.; Stützel, H. A new method for assessing the sustainability of land-use systems (II): Evaluating impact indicators. Ecol. Econ. 2009, 68, 1288–1300. [Google Scholar] [CrossRef]
  62. Walter, C.; Stützel, H. A new method for assessing the sustainability of land-use systems (I): Identifying the relevant issues. Ecol. Econ. 2009, 68, 1275–1287. [Google Scholar] [CrossRef]
  63. Meul, M.; Passel, S.; Nevens, F.; Dessein, J.; Rogge, E.; Mulier, A.; Hauwermeiren, A. MOTIFS: A monitoring tool for integrated farm sustainability. Agron. Sustain. Dev. 2008, 28, 321–332. [Google Scholar] [CrossRef]
Figure 1. Variables involved in agricultural sustainability assessment.
Figure 1. Variables involved in agricultural sustainability assessment.
Sustainability 11 05120 g001
Figure 2. Evaluation process to introduce a methodological framework for the systematic review of agricultural sustainability studies.
Figure 2. Evaluation process to introduce a methodological framework for the systematic review of agricultural sustainability studies.
Sustainability 11 05120 g002
Figure 3. Sample presentation (review papers): (a) Number of papers per year and (b) type of paper.
Figure 3. Sample presentation (review papers): (a) Number of papers per year and (b) type of paper.
Sustainability 11 05120 g003
Figure 4. Spatial coverage of the papers reviewed.
Figure 4. Spatial coverage of the papers reviewed.
Sustainability 11 05120 g004
Figure 5. Criteria frequency use.
Figure 5. Criteria frequency use.
Sustainability 11 05120 g005
Figure 6. Methodological framework for the systematic review of agricultural sustainability studies.
Figure 6. Methodological framework for the systematic review of agricultural sustainability studies.
Sustainability 11 05120 g006
Figure 7. (a) Year of publication (% percentage). (b) Geographical origination of publication (% percentage).
Figure 7. (a) Year of publication (% percentage). (b) Geographical origination of publication (% percentage).
Sustainability 11 05120 g007
Figure 8. (a) Type of data (% percentage). (b) Accessibility of data (% percentage).
Figure 8. (a) Type of data (% percentage). (b) Accessibility of data (% percentage).
Sustainability 11 05120 g008
Figure 9. Crop agricultural sustainability at farm level (method categories and methods).
Figure 9. Crop agricultural sustainability at farm level (method categories and methods).
Sustainability 11 05120 g009
Table 1. Review studies examined.
Table 1. Review studies examined.
a/aReferenceType of ReviewSpatial Coverage
1De Luca, A. I., et al., (2017) [10]Critical and SystematicGlobal
2Binder, C.R., et al., (2010) [4]CriticalGlobal
3Peter, C., et al. (2017) [12]SystematicGlobal
4Bockstaller, C., et al. (2009) [3]CriticalGlobal
5Roy, R., et al. (2012) [13]SystematicCountry level - Bangladesh
6Cerutti, A.K., et al. (2011) [7]SystematicGlobal
7Acosta-Alba, I et al. (2011) [14]CriticalGlobal
8Baldini, C., et al. (2017) [15]Critical and SystematicGlobal
9Morais, T.G., et al. (2016), [16]SystematicCountry level - Portugal
10McAuliffe, G.A., et al. (2016) [17]SystematicGlobal
11Latruffe, L., et al. (2016) [18]CriticalGlobal
12Bockstaller, C., et al. (2008) [19]CriticalGlobal
13Payraudeau, S., et al. (2005) [20]SystematicGlobal
14de Vries, M., et al. (2015) [21]SystematicGlobal
15Yan, M.-J., et al. (2011) [9]SystematicEurope
16Lebacq, T., et al. (2013) [22]CriticalGlobal
Table 2. Research keywords and inclusion/exclusion criteria.
Table 2. Research keywords and inclusion/exclusion criteria.
AgricultureSustainability AssessmentPrimary ResearchReview
FarmingTriple Bottom Line AssessmentIn EnglishNot in English
AgriculturalAgricultural SustainabilityPublished from 2009Published before 2009
FarmEnvironmental AssessmentPeer-reviewedBook Chapters
LivestockEconomic AssessmentAgriculture /LivestockConference Proceedings
HusbandrySocietal/Social AssessmentPrimary ProductionGrey literature
TillageLife Cycle Assessment (or LCA)
AgronomyMulticriteria Decision Analysis (MCDA)
StockraisingIndicators (or KPI)
Environmental Impact Assessment (or EIA)
Cost–Benefit Analysis (CBA)
Table 3. Initial screening (38-paper collection).
Table 3. Initial screening (38-paper collection).
a/aAuthorCountryMethod/ToolMethodology/ApplicationField of Application/ProductLevel of Assessment/System Boundaries
1De Luca et al. (2018) [31]ItalyLCSA AHP LCA LCC SLCAM, AOliveF
2Snapp et al. (2018) [32]MalawiINDICATORSM, AMaize, legumeF
3Gaviglio et al. (2017) [25]ItalyINDICATORS
M, AAgricultural ParkF
4Recanati et al. (2017) [33]PalaistineINDICATORSM, AFood ProductionF, R
5Bockstaller et al. (2017) [34]-CONTRAM, A-F
6Goswami et al. (2017) [35]-SFSI DPSIR SLMSmal FarmsF
7Theurl et al. (2017) [36]AustriaLCA CF INDICATORSALettuce, spinach, scallions, red radishF, R
8Chopin et al. (2017) [37]FrenchWest IndiesMASCABananaF, N
9Angevin et al. (2017) [26]-DEXiPMM-F
10Vasileiadis et al. (2017) [38]EuropeDEXiPM SYNOPS-WEB CBAAWheat, maizeF, R
11Egea et al. (2016) [39]SpainMCDA ANPMOliveF
12Dong et al. (2016) [40]USAPCA DEAM, ASoybeanF
13de Olde et al. (2016) [28]DenmarkRISEAOrganic Farms (vegetable, dairy, pig, poultry)F
14Yang et al. (2016) [29]ChinaINDICATORSM, AGreenhouse vegetablesF
15Sajjad et al. (2016) [41]IndiaSLSIA-F, R
16Allahyari et al. (2016) [42]IranINDICATORSMPaddy fieldsF
17Sottile et al. (2016) [30]KenyaSAEMETH-G PCAM, ASchool GardensF
18El Chami et al. (2015) [43]EnglandLCA CBA SCCM, AWinter WheatF
19Santiago-Brown et al. (2015) [44]-INDICATORSMGrapesF
20Peano et al. (2015) [45]-SAEMETHM, A10 small agri-food systemsF
21Dong et al. (2015) [46]USAPCA DEAM, ACranberryF
22Yegbemey et al. (2014) [47]BeninINDICATORSM, AMaizeF
23Peano et al. (2014) [48]ItalyINDICATORSM, AAgri-food system Slow food Presidia ProjectF
24Van Asselt et al. (2014) [49]The NetherlandsINDICATORSM, APotatoF
25Colomb et al. (2013) [50]FranceMASC-OF DEXiM, A-F, R
26Vasileiadis et al. (2013) [51]EuropeDEXiPMAWheat, maizeF
27Sami et al. (2013) [52]IranINDICATORSM, AWheat maizeF
28Pelzer et al. (2012) [27]FranceDEXiPMM, AArable crops and maize-based systemsF
29Van Passel et al. (2012) [53]BelgiumSVA MOTIFSM, A-F
30Reig-Martinez et al. (2011) [54]SpainDEA MCDAM, A-F
31Sharma et al. (2011) [55]IndiaASIM, A-F
32Rodriguez et al. (2010) [56]-APOIA-NovoRuralM, A-F
33Gomez-Limon et al. (2010) [24]SpainSAFE AHP PCAM, ARain-fed and irrigatedF
34Gomes et al. (2009) [57]BrazilDEAARice, maize, coffeeF, R
35Van Passel et al. (2009) [58]BelgiumSVAM-F
36Sadok et al. ([59]FranceMASC DEXiM, ACropping systemsF
37Siciliano (2009) [60]ItalySMCEM, ADurum WheatF, L, R
38Walter et al. (2009) [61,62]GermanyINDICATORSM, ASpinachF, R, G
M: Methodology; A: Application; F: Farm level; R: Regional level; N: National level; G: Global level.

Share and Cite

MDPI and ACS Style

Lampridi, M.G.; Sørensen, C.G.; Bochtis, D. Agricultural Sustainability: A Review of Concepts and Methods. Sustainability 2019, 11, 5120.

AMA Style

Lampridi MG, Sørensen CG, Bochtis D. Agricultural Sustainability: A Review of Concepts and Methods. Sustainability. 2019; 11(18):5120.

Chicago/Turabian Style

Lampridi, Maria G., Claus G. Sørensen, and Dionysis Bochtis. 2019. "Agricultural Sustainability: A Review of Concepts and Methods" Sustainability 11, no. 18: 5120.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop