“Doing” Sustainability Assessment in Different Consumption and Production Contexts—Lessons from Case Study Comparison

Abstract

Sustainability as a guiding idea for societal and economic development causes a growing need for reliable sustainability assessments (SAs). In response, a plethora of increasingly sophisticated, standardizAed, and specialized approaches have emerged. However, little attention has been paid to how applications of SAs in different contexts navigate the challenges of selecting and customizing SA approaches for their research purposes. This paper provides an exploration of the context-specific conditions of SA through a case study of three research projects. Each case study explores the different approaches, methodologies, as well as difficulties and similarities that researchers face in “doing” SA based on the research question “What are common challenges that researchers are facing in using SA approaches?” Our case study comparison follows a most different approach for covering a wide range of SA applications and is structured along with three key challenges of doing SA: (i) Deliberation, learning and assessment; (ii) normative assessment principles; (iii) feasibility, especially regarding data quality/availability. Above all, the comparative case study underlines the role and importance of reflexivity and context: We argue that a more explicit and transparent discussion of these challenges could contribute to greater awareness, and thus, to improving the ability of researchers to transparently modify and customize generic SA methodologies to their research contexts. Our findings can help researchers to more critically appraise the differences between SA approaches, as well as their normative assumptions, and guide them to assemble their SA methodology in a reflexive and case-sensitive way.

1. Introduction

Sustainability is widely recognized today as a guiding idea for societal and economic development. With the growing importance of sustainability, there is also a growing need for reliable and comparable approaches that allow measuring the sustainability of a given object, e.g., a product, service, process, or an enterprise. In response to this demand, a plethora of approaches [1,2] has emerged to provide sustainability assessments (SAs). The SA field has origins in previously established methods like industrial ecology, LCA [3], policy impact evaluation [4], and has been increasingly established and institutionalized in the form of political commissions and declarations, such as the Commission on the Measurement of Economic Performance and Social Progress [5] and the Istanbul Declaration [6] and “standards” like Bellagio STAMP [5]. Spurred by research, the sophistication, standardization, and specialization of SA approaches are improving. However, little attention has been paid to how research projects navigate the challenges of selecting and customizing SA approaches for their unique research purposes.

In considering general aspects of sustainability assessments, it is important to note that there is no interdisciplinary approach to identifying standards nor a consistent basis for SA [7]. The field of SA is highly diversified, cuts across sectors, products, processes and lifestyles/consumption patterns so researchers need to know what sustainable practices are, products, firms, sectors, policies, countries. Sustainable development is multidimensional and complex, so it is not possible to have a single indicator; furthermore, SA is informed by different understandings of sustainable development, which makes the operationalization of sustainable development and SA more complicated. Moreover, SA has to face the normative content of sustainable development: When trying to design and implement sustainable development, sooner or later there are conflicts of interest that have to be dealt with, but dealing with tradeoffs can hardly be addressed by science. This means that the plurality of approaches to SA can be meaningful because of the very different tasks, goals, subjects, objects, reference levels and contexts of SA on the one hand and because of normative assumptions and underlying values as we work out in the study.

Nevertheless, SA is something that most researchers in sustainability have to “do”. In the course of our respective projects, we identified a gap in the literature when it comes to a practice-oriented understanding of the main challenges that researchers face when they engage with SA in diverse contexts. This led to our research question “What are common challenges that researchers are facing in using SA approaches?”

In the past decades, a wide variety of SA tools has been produced [8]. In this context, the vague and diverse definitions of sustainability itself remains a problem. There are various sustainability assessment tools that not only consider different perspectives or indicators but also differ in their methodology. As such, the following attempt to define characteristics for sustainability assessments can only be rather generic.

According to Sala et al., “Sustainability assessment (SA) is one of the most complex types of appraisal methodologies. Not only this does entail multidisciplinary aspects (environmental, economic and social), but also cultural and value-based elements. Besides, SA is usually conducted for supporting decision making and policy development in a broad context” [9].

Additionally, the wide field of SA makes it an easy target for lawmakers, companies or other interest groups to pride themselves with implementing or acting upon sustainable assessments or tools, without the capability of measurement or adequate comparison. Still, there is a certain interest in keeping the definition of sustainability as broad as it is, in order to keep the great variety of stakeholders and their areas of action included in the discourse. As Pope et al. puts it: “[T]he concept of sustainability has the potential to not only keep everyone at the table, but to provide the catalyst for reflexivity and a deliberative space, or axis around which discussion can occur” [8].

1.1. Goals of Sustainability Assessments

Sustainability assessments can, thus, only be defined in a somewhat generic manner. Likewise, the goals and targets of SA follow a broad approach.

One definition of overall goals in sustainability assessment is provided by Verheem: “The goal of sustainability assessment is to pursue that “plans and activities make an optimal contribution to sustainable development” [10]

A more elaborate outlook is given by Waas et al. on how SA support decision-making by addressing four purposes:

• Substantive: Describing “the achievement of the intended purposes of the SA”;
• Normative: Focusing on “the achievement of normative goals—i.e., can stakeholders learn, improve their knowledge and change their views”;
• Procedural: “[C]onsideration of SA process aspects and the establishment of SA, procedures and policy”;
• Transactive: “[T]he achievement of intended purposes with minimal resources and time or in other words efficiency” [11].

1.2. Precise Subject of Sustainability Assessments

The conceptual framework can be established with two main dimensions and various sub-dimensions, one being the sustainability concept (including social, economic and ecologic aspects) and the other the decision-making context. The decision-making context is particularly important to bring the theory of sustainability into practice and involve the different stakeholders identified [8,9]. However, there is no general conformity about the types and quantity of dimensions to encompass the SA.

When deciding about developing an ex ante or ex post tool, a helpful differentiation might be to review whether the means of the assessment tool are to guide and direct decision-making by predicting effects before the implementation. If the technique rather requires a retracing look upon things, so, for example, the assessing of processes, an ex post tool is required. According to many authors in this field, to design sustainability assessment in the context of decision-making, an ex ante tool is the most common approach [8].

When defining the precise subject of sustainability assessment, there are different frameworks that can be considered. The different SA tools can be categorized as product-related assessment tools, project-related assessment tools and country-related assessments, as well as an indicator-related approach. The following

Table 1. Sustainability assessment tools, taken and slightly edited from Reference [12] based on the overview with all listed tools [13].

	Indicators/Indices	Product-Related Assessment	Project-Related Assessment	Sector and Country-Related Assessment
Environmental	- Environmental Pressure Indicators (EPIs) - Ecological Footprint (EF)	- Life Cycle Assessment (LCA) - Material Input per Service (MIPS) Unit - Substance Flow Analysis (SFA) - Processes energy analysis - Exergy analysis - Emergy analysis	-Environmental impact assessment (EIA) - Environmental Risk-Analysis (ERA)	- Environmental Extended Input-Output (EEIO) Analysis - Input-Output Energy Analysis - Strategic Environmental Assessment (SEA) - Regional emergy analysis - Regional exergy analysis
Economic	- Gross National Production (GNP)	- Life Cycle Costing (LCC)	- Full Life Cycle Cost Accounting (FCA)	-Economy-Wide Material Flow Analysis (EW-MFA) -Economy wide substance flow analysis - Economic Input-Output (EIO) analysis
Social	- Social Indicators		- Social Impact Assessment (sIA)	- Social Input-Output (SIO) analysis
Sustainable Integrated Development	- Human Development Index (HDI) - Environmental Sustainability Index (ESI) - Wellbeing Index (WI) - Sustainable National Income (SNI) - Genuine progress indicator (GPI), ISEW, Genuine Savings - Sustainable Development Indicators (SDI) - Sustainable energy development indicators (SEDI)		- Cost-Benefit Analysis (CBA) - Risk Analysis (RA)	- Multi-Criteria Analysis (MCA) - Uncertainty analysis - Vulnerability analysis - Conceptual modelling - System dynamics - Sustainability Impact Assessment (SIA) - Integrated Sustainability

illustrates these categories and gives examples for established tools.

Table 1 lists a wide range of tools. Some of the tools from the environmental, the economic and the ecologic sections can be seen as sources or methodological foundations for integrated tools addressing sustainable development or as parts of sustainability indicator sets (e.g., Ecological Footprint, LCA or MIPS). A couple more tools, especially in the non-existing column for enterprise-related assessment, could easily be added (diverse management systems, as well as self-assessment tools, for an overview, see Reference [14]). Irrespective of the existing, multi-faceted instruments, the projects involved in the study had to “tailor” their tools to their respective contexts as described in Section 3.

The assessment of sustainability impacts is a complex scientific issue that immediately becomes normatively loaded. For that reason, reflective and critical discussions, as well as exchanges on strategies are needed to further develop scientific research and practical experience and to obtain greater comparability. This paper addresses the gap resulting from the little attention that has been paid to how research projects navigate the challenges of selecting and customizing SA approaches for their unique research purposes.

In that context, a case study comparison can make a contribution by developing approaches for the subject of SAs (structuring diversity; generating comprehensive results, comparison of different key aspects in the field of sustainability) and identifying academic gaps.

Therefore, this paper provides relevant input for a discussion that needs to be further scientifically investigated, by serving as a methodological basis while specific strategies are presented and comparatively discussed.

It provides an exploration of the context-specific conditions of SA through a comparative case study design of three research projects within a German research program on the sustainable economy. Each case study explores the different approaches, methodologies, as well as difficulties and similarities that researchers face in “doing” SA.

Against this background, the following research questions guided the comparative analysis:

Are there similarities and similar difficulties in doing SA in different settings and projects and how can we learn to overcome the diverse challenges in SA? By answering the research question, we aim to create awareness and to improve the ability of researchers to transparently modify and customize generic SA methodologies to their research contexts.

To address our research questions, it was important to take a view beyond the horizon and analyze different projects that deal with SA in various contexts regarding their SA goals, lessons learnt, etc., to see if there is “one best way” to do SA. We develop our investigation along a comparison of three different projects.

Section 2 describes the methods and materials used, where we detail the comparative case study design, the research context and the analyzed projects. In Section 3, we then present the results obtained from applying the methods for each of the three cases compared. Based on a comparison of the results, we discuss the main commonalities and differences in Section 4, before we briefly summarize our findings in Section 5.

2. Materials and Methods

Most Different Case Study Approach

To answer the question about similarities and challenges in doing SA, we compared three different projects which dealt with SA in different contexts. We hereby aim to assess whether or not SA-related challenges and pathways depend on project goals; and to find which aspects of SA affect every SA researcher:

The most different case study approach is inspired by a comparative case study that would attend simultaneously to macro, meso, and micro dimensions of case-based research. The approach engages two logics of comparison: First, the more common compare and contrast; and second, a “tracing across sites or scales” [15]. The comparative case study approach can be considered as a specification of the case study research [16]. It is suitable “to formulate or assess generalizations that extend across multiple cases” [17]. However, with its focus on “doing” SA, in our study, we do not claim to connect all levels, to work out generalizations, etc., but our most different approach is sufficient for the purpose of working out similar challenges of SA in different contexts. Moreover, as the overarching project “NaWiKo” (see research context in the next subsection) aims to attempt a synthesis of findings and methods in the field of a sustainable economy, this most different approach in SA challenges with its diversity covered by the three projects is appropriate.

At this point, we delineate ourselves from a positivistic claim to replicability. Replicability was not the goal, which is why we tried to generate dense descriptions, interior views and reflections from the projects, enabling other researchers to reflect transparently on how they would like to customize generic SA methodologies to their contexts. This does not need to be replicable per project, but the steps of considerations should be and are replicable—i.e., the steps we have chosen to work on results in Section 3.

For each of the three case studies, the methodology applied within each project was examined by looking at external (context) features and internal (design) features. Since goals and subjects of the assessment are set by the purposes of the respective projects in their common funding context, these can be seen as context features from the assessment perspective. The internal (design) perspective, on the other hand, is shaped by the consideration of the steps in the course of the assessments.

This paper focusses on the following three projects: NAHGAST (sustainable out-of-home catering), BiNKA (education for sustainable consumption through mindfulness training), and REFOWAS (pathways to reduce food waste). These projects are currently in the phase of evaluation, utilization or extension. The present paper covers three of 30 research projects within the funding line FONA (Research for Sustainable Development of the SÖF (Social-ecological Research) funding program. These projects are coordinated through the overarching project NaWiKo. We have chosen the three projects because of their comparability in conceptualizing and implementing SA and because their differences in doing so represent a remarkable diversity with a wide range of goals, subjects, steps and learnings.

3. Results

3.1. Overview

3.1.1. Research Context of the Analyzed Projects

All three projects assessed are embedded in the context of sustainability sciences. This research field aims to better understand how natural and social systems interact and which approaches influence and transform such systems towards a more sustainable state [11,12,18]. Its mode of research and knowledge production seeks to take interdisciplinary perspectives and knowledge into account as well to involve non-academic actors in the research process [19].

Within this field of research, the present article can be assigned to the research area of sustainability evaluation/sustainability assessment that aims to develop “consistent and robust indicators and methods” and to address the “challenge [of] a comprehensive, yet understandable presentation of the results” [20].

3.1.2. Subjects and Objectives of the Sas in the Three Projects under Study

In

Table 2. Focus and objectives of the sustainability assessments (Sas) of the three compared projects (source—own work).

Project	Fostering Sustainable Food Out of Home (NAHGAST Project)	Fostering Sustainable Mindsets (BiNKA Project)	Fostering a Sustainable GERMAN Food Sector (REFOWAS Project)
Focus	Development of a tool for kitchen professionals to determine the sustainability performance of their products—the offered meal.	Mindfulness meditation intervention designed to promote sustainable consumer behavior among participants.	Analysis of the German food sector as regards to reasons for food waste. Identification and testing of strategies and preventing options to reduce food waste. Development of a complex hybrid-LCA model of the German food sector to quantify environmental indicators.
Objective	Assess the sustainability performance of a single meal which is a different starting point than assessment tools that monitor, e.g., the overall processes of a company. The assessment of the single meal revolves around one product, which is related to multiple processes in- and outside of companies that all play a part in the life-cycle of this meal.	Detect changes in individuals’ consumer behavior (more frequent/rare occurrence of sustainable behavior) using a questionnaire	Understand the food supply chain in terms of sustainable production and consumption.

, we compare the subjects and objects of the three projects.

The projects pursue distinct overriding purposes, resulting in different starting points for sustainability assessment.

The NAHGAST-tool assesses the sustainability of meals and should ideally result in a significant sales increase in sustainable meals. These meals first need to be produced, which is why clarity for producers about sustainability is needed. The target audience of the tool is, therefore, producers, as well as consumers, even though the initial position is clearly on the side of the producers.

In contrast, the purpose of BiNKA was to contribute to sustainable consumption by acting attentively. The focus, in this context, is clearly on the consumers.

On the other hand, the purpose of REFOWAS was to find reasons why food waste is produced, to identify measures to reduce food waste, and to measure environmental impacts that are related to overall German nutrition. The focus here is on both the consumers and the producers. In addition, the political level is addressed.

In accordance with the projects’ funding context, the three projects do not remain on the analytical level, but also pursue a normative approach to promote the sustainable production and/or consumption. The extent of this aspect of intervening, however, varies across the projects.

Apart from the normative approach for sustainable development, the three projects focus on very different subjects and use various individual methods, as will be exemplified in the following sections. Furthermore, we will illustrate the variety of fields of applications in which sustainability assessment can be useful in the context of sustainability research. In addition, we try to make it clear that a comparison of the various approaches is useful, as well as logical, even though the concepts are very different.

3.2. NAHGAST: Characteristics of SA

3.2.1. External (Context) Features—Goals and Subjects

The NAHGAST-Assessment tool was primarily developed to enhance sustainability in the out-of-home catering sector [21,22,23]. Further overarching goals and aims regarding the assessment tool were formulated as follows:

• The NAHGAST-tool should ideally result in a significant sales increase in sustainable meals
• The tool aims at supporting to compare and contrast the impact of the used ingredients of a single meal on sustainability dimensions in the wider life-cycle context between different meals.
• The objectives a company has and uses the tool for are not predetermined by the tool and manifold ways of making use of the results are possible and suggested.

In the development of the tool, the user environment of canteen workers was considered in order to eventually make the assistance of scientists and consultants obsolete and enable practitioners to add all necessary data themselves. All in all, the NAHGAST is, therefore, a valuable addition to research carried out in the effort of sustainability assessment

Sustainability, as the overarching effort, is not in itself a category that can be operationalized with definite figures, and therefore, needs to be structured and flanked by indicators. Accordingly, the core of the NAHGAST SA is the definition of suitable indicators, the determination of sustainable levels for a meal-related to the respective indicators and the calculation of food offers in terms of defined indicators and target values.

3.2.2. Internal (Design) Features—Methods and Steps

In order to develop the tool, an understanding of how to select the specific indicators underlying the broad definition of sustainability had to be achieved. In NAHGAST, the main criteria for the selection of indicators were: Communicability, Feasibility, Scientific Relevance. Communicability implies how easily practitioners and consumers are able to comprehend a certain indicator. Feasibility makes sure the information to be entered can be easily obtained by kitchen staff. Expected expenses and human resources for preparing and entering the data were also relevant for this criterion. To serve the overall goal to increase sustainability in single meals, scientific relevance was a crucial point in the selection of indicators. The selection of indicators in NAHGAST was made with the aim of choosing “objectively” important sustainability topics. Nevertheless, any selection of indicators always has a “subjective” component.

The information the indicators contain can help a company to define their goals or, in the NAHGAST-example to “decrease material/carbon footprint of meals”, “improve the score on the social dimension of sustainability”, “increase fiber content of meals”, etc. From the unit of the single meal, the relevant processes assessed are related to the wider context: Ecological aspects, social aspects, health aspects. Additionally, economic aspects have been taken into account for NAHGAST.

The target values for the different indicators (including a transition area) were defined, in order to have a benchmark that serves as the basis for evaluation. The defined target values refer to a healthy person with an average energy consumption of 2000 kcal per day.

Table 3. Target levels NAHGAST tool (Source: Edited from Reference [22]).

Indicator	Target Value (Range of Tolerance)	Source
Carbon footprint	<800 (1200) (g CO2eq) per meal	Lukas et al. 2016 [23]
Water demand	<640 (975) l per meal	Lukas et al. 2016 [23]
Material footprint	<2670 (4000) g per meal	Lukas et al. 2016 [23]
Demand for land	<1.25 (1875) m2 per meal	Lukas et al. 2016 [23]
Energy content	<670 (830) kcal per meal	Lukas et al. 2016 [23]
Fiber content	>8 (6) g per meal	Lukas et al. 2016 [23]
Salinity	<2 (3.3) g per meal	Lukas et al. 2016 [23]
Content of carbohydrate	<90 (95) g carbohydrates per meal	DGE 2011c [24]
thereof sugar	<17 (19) g sugar per meal	DGE 2011c [24]
Fat content	24 (30) g fat per meal <6.7 (10) g saturated fat	Lukas et al. (2016), DGE 2014, [23,25]
Share of animal products from species-appropriate environment	60% (55%) of animal products	Own estimation based on multi-stakeholder dialogue [23]
Share of fair-trade products	90% (85%) of products from developing and emerging countries with fair trade label	Own estimation based on multi-stakeholder dialogue [23]

shows the target levels for the NAHGAST tool. The stated target values refer to a healthy person with an average energy consumption of 2000 kcal per day. We recognize that other sections of population like children, or elderly people exhibit other needs, and therefore, other target values would apply.

A well-grounded transdisciplinary research approach allowed an in-depth analysis of kitchen processes and the determination of requirements and challenges specific for kitchen settings. Implemented in cooperation with several catering companies, the tool was constantly revised to improve its consistency, as well as its user-friendliness in and relevance for canteen operations.

The tool can serve either as an internal monitoring instrument or as an instrument to generate information about meals for the end customer. In carrying out the assessment, the data on meal ingredients are inserted in an excel spreadsheet. The results are generated automatically and a rating as “recommendable”, “restrictively recommendable” or “not recommendable” appears. Each sustainability dimension (ecology, social, health, economy) is composed of several indicators which form a combined rating, and the meal also receives an overall score.

After the assessment has been carried out, the results can be used in manifold ways. It was a vital part of the design of the tool to display the results in an appealing manner, keeping in mind the option to utilize the tool for marketing purposes. The NAHGAST tool shows the impact of the used ingredients on the different sustainability dimensions in the wider life-cycle context for a single meal in the form of a score. Additionally, the NAHGAST results can be translated into a label Figure 1 exemplifies.

The findings can be used for further interventions with customers, e.g., “nudges” (like the alternative presentation of food), translation into a label, and communication of sustainability-related information on the meals in order to find out how the customers react towards the single assessed meal. It is now possible to compare the impact of the used ingredients on sustainability dimensions in the wider life-cycle context between different meals. The meal assessment can lead to an optimization of the ingredients of one meal, in order to minimize the negative impact of the used ingredients on sustainability dimensions in the wider life-cycle context. This can happen by: (a) Changing the qualities of ingredients, (b) changing recipes, or (c) changing the menu.

3.3. BiNKA: Characteristics of SA

3.3.1. External (Context) Features—Goals and Subjects

The overall goal of the sustainability assessment in the BiNKA project was to construct a questionnaire assessing the sustainability of individuals’ consumption behaviors in two different consumption areas (food and clothing). This measure was then used to analyze the effects of the intervention (mindfulness training) on participants. Individual consumption behaviors were selected according to three dimensions: Consumption area (food and clothing), consumption phase (acquisition, usage and disposal) and sustainability impact (ecologic and socio-economic). Within the construction of the assessment instrument (questionnaire), those behaviors with the highest impact were considered.

At the heart of the BiNKA project was an intervention study that focused on the effects of a consumption-specific mindfulness training on participants. Hence, the subject of assessment was individual consumption behavior. In the wider sustainability context, the assessment involved ecological, social, health-related, and economic indicators.

3.3.2. Internal (Design) Features—Methods and Steps

In the assessment procedure, the sustainability of individual consumption behaviors were defined in an impact-oriented perspective by the effects of the behaviors on outer ecological (planetary boundaries) and inner socio-economic (minimum social standards) conditions, as suggested by the Doughnut model [26]. As argued elsewhere [27], both conditions need further concrete criteria to measure the sustainability impacts of consumer behaviors. An influential approach to assess ecological conditions is the planetary boundaries framework, which identifies nine essential life support systems [28,29]. Another prominent approach to assess ecological impacts is footprint methodology. This has been used to calculate general ecological [30], carbon [31] or integrated footprints [32]. Possibly, even more diverse and controversial are approaches that attempt to measure and evaluate impacts on the socio-economic dimension of sustainability [33]. Two much-discussed approaches on the question of what conditions people need, in order to thrive and lead a good life, are the capability approach [34] or matrices of objective or fundamental human needs [35]. These serve as the basis to derive concrete indicators, such as safety, food security and household goods (living standards) or jobs and income (livelihood) [36].

Table 4. Relationship between sustainability dimensions and behavioral items with examples for each level.

Sustainability Dimension	Ecologic		Socio-Economic
Theoretical Approach	Planetary Boundaries	Ecological Footprint	Capability Approach	Fundamental human needs
Criteria	Climate change	Land use	Poverty	Livelihood
Indicator	CO2	km2 of arable land	Income	Income
Behavioral Item	I buy imported fruits from overseas.	I buy organically grown food.	I buy fair trade food products.	I buy fair trade food products.

provides an overview of how sustainability impact dimensions, theoretical approaches, criteria, indicators, and specific behaviors are interrelated.

The assessment tool was developed in four steps (as described in Reference [27]): The definition of (1) the sustainability dimension(s), (2) consumption area(s), and (3) consumption phase(s) of interest, and finally the (4) selection of consumption behaviors with the highest impact in the chosen dimension/area/phase. We focused our assessment on the two boundaries of ecologic and socio-economic sustainability [26] (Step 1). Both dimensions have been addressed in the intervention. With regard to consumption areas, we focused on food and clothing (Step 2). Our assessment included the consumption phases (Step 3) of acquisition, usage, and disposal. To select those behaviors with the greatest sustainability impact for each consumption area and phase (Step 4), we reviewed items used in other scales [37,38], and classified them according to the sustainability dimensions of Step 1. In the following, we present how we used this approach to identify high-impact behaviors for food consumption (as presented in greater detail in Reference [27]).

As core indicators of ecological impacts, we chose energy consumption, GHG emissions and ecological footprint. The most important behaviors were the consumption of meat, dairy products, and organic and non-imported foods [39,40,41]. The most relevant impacts of storing and preparing food products were cooking techniques, like using pot lids [42], and the avoidance of frozen ready-made meals [43]. Finally, single use food packaging and food losses were identified as key points in the disposal phase [44,45,46,47].

Core indicators for socio-economic sustainability impacts are working conditions and fair prices, health issues and fair distribution. Consumers’ purchase choices can support fair working conditions and decent income for producers, e.g., through fair trade [48,49]. Other high-impact domains in the socio-economic dimension are to strengthen small-scale economies and food security by supporting regional food provision systems or local alternative food networks and growing food for self-supply [50,51,52]. Concerning the usage of food, there is evidence that it can have a positive effect on people’s self-confidence and independence and strengthen cohesion between household members when meals are prepared within the household [53]. In addition, healthy diets are seen as a contribution to individual life-satisfaction and reduced social costs [54,55]. Finally, a high-impact behavior for the disposal phase is to purchase food products close to or beyond the minimum expiry date [56].

The final scale developed comprises 16 items for both sustainability dimensions, which were assessed on a general 7-point Likert scale from 0 = “never” to 6 = “always” for items that concern non-daily activities like shopping. For daily activities focused on eating and cooking practices, we adapted the scale option from 0 = “never” to 6 = “daily” with the middle option reading 3 = “once a week” [27].

3.4. REFOWAS: Characteristics of SA

3.4.1. External (Context) Features—Goals and Subjects

The overall goal of the sustainability assessment in the REFOWAS-Project was to analyze the ecologic aspects concerning avoidable food waste in the value chains of the agriculture and food sector in Germany. Hence, a consistent and complete model of the German food sector, including food waste, was created. Special attention was paid to quantifying the avoidable fraction of food waste, evaluating its impacts and developing and testing of waste reduction measures and strategies. Thus, an integrated sectoral analysis of the overall system was complemented by more detailed case studies for certain subsections (fruit/vegetables, bakery products, school catering). Thereafter, the potentials of avoiding food waste were analyzed in case studies, for example, the ecological effects for meeting the Sustainable Development Goal 12.3 of the United Nations (SDG 12.3: Half of the foods waste in consumption and handling till 2030). The model shows an overview of the status quo of the German food consumption, including food waste, and generates possible benchmarks for twelve German food product groups to monitor the achieved results and analyze the ecological effects of further approaches to reduce food waste.

The researched area within REFOWAS-Project was the German food sector and its food wastes. Therefore, the German food sector was examined (for the data sources and steps undertaken see Section 3.4.2) and divided into four subsectors (Agriculture, Processing, Wholesale and Retail, Consumption).

3.4.2. Internal (Design) Features—Methods and Steps

First, twelve product groups were defined:

• Meat and meat products;
• Eggs and egg products;
• Milk and milk products;
• Grain and grain products;
• Potatoes and potatoe products;
• Other foods (e.g., spices, sauces);
• Oils and fat;
• Sugar and sweets;
• Vegetables and vegetable products;
• Fruits and fruit products;
• Fish and fish products;
• Beverages (tap water, lemonade, coffee, tea, alcoholic beverages, etc.).

For those groups and each of the subsector, results can be calculated and used, e.g., as benchmarks.

The choice of indicators was based on criteria, such as comparability, prominence, common usage, feasibility and reliability. For feasibility and reliability, an extensive basis of data, scientific values and knowledge on specific topics are required. This shifted the focus to the global warming potential (GWP), the cumulative energy demand (CED) and the agricultural land use. The results for the twelve product groups are given as coefficients per kilogram of the product (kg CO₂-eq/kg, MJ/kg, m²*a/kg).

More indicators could be added in future studies, but the database for further indicators is not given at the moment. Within the academic literature, the focus concerning the topic of food waste is often placed on the GWP or land use [57]. Therefore, the choice of indicators seems reasonable. These coefficients/benchmarks were the basis of the study.

For the system analysis, a representative and the consistent database have been created, in order to describe and balance the system of food production and consumption. Mass and energy flows have been displayed based on existing data, as well as the results from the REFOWAS case studies, consuming studies and waste sorting analyses. The main data have been taken from the statistical yearbook of the BMEL (German Federal Ministry of Food and Agriculture), from the short-term statistics of the manufacturing sector, as well as from import and export statistics [58,59,60]. Data regarding consumption are taken from the income and consumption sample [61], the national consumption study II [62] and further consumption studies [63,64], and extrapolated for the German population. Food waste volumes were taken from literature and monitoring data of the University of Stuttgart [65]. The differentiation between unavoidable and avoidable food waste is also based on factors created by the University of Stuttgart [65,66].

By using this data, the German food sector was first characterized in terms of how much is consumed and produced, imported and exported and how much food is lost or wasted. Therefore, the coefficients mentioned in Section 3.3.2 were extrapolated with the consumption of the German population, to receive results that reflect the situation in Germany. To balance the model and to close data gaps, the amount of tap water which is added or deducted, depending on the process step, was integrated into the calculation, according to factors of weight changes [67]—for example, water absorption by dry pasta or water evaporation by mushrooms during cooking. This is particularly important for evaluating the environmental impacts associated with food loss measures. The difference between the dry mass of the raw materials and the wet mass of the waste products leads to a distortion of the results, when water is neglected.

In a second step, environmental impacts associated with to the entire German food sector (production, processing, distribution, consumption and waste generation) were quantified using environmental information from different studies and life cycle assessment databases within the Life Cycle Assessment (LCA) software openLCA (http://www.openlca.org/) [68]. The main data sources of the environmental information are:

• Agricultural impacts: Schmidt and Osterburg (2010) [69];
• Distribution, processing and out-of-home sector: Destatis [59,60,61];
• Imports: EXIOBASE (www.exiobase.eu) (Wood et al. 2015, Tukker et al. 2013) [70,71];
• Private households: Sima et al. (2012) [72], and BVEW (Bundesverband der energiebezogenen Wirtschaft) (2013) [73];
• Background and prechain processes: Ecoinvent 3.3 (www.ecoinvent.org) (Wernet et al. 2016) [74];
• Additional sources for transport processes: Keller (2010) and BMEL (2013) [75,76].

A combination of a bottom-up and a top-down approach (Hybrid-LCA) provided the best results. Therefore, disaggregated sectoral data and also processed data were used to achieve a preferably balanced and complete model.

A hot spot analysis was conducted in order to identify the main sources of waste generation. Additionally, scenario analyses (e.g., ecological effects regarding the SDG 12.3), the regional distribution of the global land use of the German food consumption and differences between animal products, plant products and beverages followed the basis calculation. Finally, sensitivity analyses, analyses regarding the uncertainties of the model and classification of the results compared to literature were added.

3.5. Key Lessons

3.5.1. Key Lessons Learnt NAHGAST

The NAHGAST-tool combines very different indicators; sustainability indicators are combined with nutritional values, and quantitative and qualitative measures are put into relation. This proved to be a problem in designing the tool consistently, as the question appears whether it really is possible to juxtapose ecological, nutritional, social and economic indicators or whether they should remain in different categories. Although the indicators were selected on a scientific basis, the selection of indicators still holds some subjectivity. Furthermore, within the field of sustainability, there are different measurement categories and standards. Nutrition standards are provided by the German Nutrition Society (DGE). However, these nutritional recommendations do not refer to single meals, but rather to what consumers eat in a day or week. Another problem the project team encountered was data-quality. In some cases, there was no data for the ecological indicators available at all, and often there was no specification in terms of organic, cultivation, origin and more. It is, therefore, necessary to revert to comparable ingredients in some cases.

The NAHGAST-tool was designed on a scientific basis, and although handling for the kitchen staff is considered as a priority, the tool remains rather complex, and for some of its features and functions, an explanation by scientists is necessary. Further improvements, especially regarding the inclusion of the tool in the management process of kitchens, is, therefore, required. An important step in the applicability of the NAHGAST-tool is the development of an online tool, which will be developed in the continuation of the project further.

3.5.2. Key Lessons Learnt BiNKA

While we consider our approach of designing measures of sustainable consumption behaviors to be more straightforward, in terms of transparency and accountability, than other measures, we are also aware of prevailing shortcomings. Two shortcomings are particularly striking. Firstly, the items in the SCB measure are chosen for their relative impact and fail to determine an absolute consumption limit considered sustainable on a per capita level, e.g., suggested by per capita budget approaches [77]. Secondly, the identification of high-impact consumption behaviors was severely impeded by the lack of comprehensive data available on consumption-related sustainability impacts on a behavioral level. To systematically include high impact consumption behaviors in the social sciences, an inventory summarizing the impact in some chosen indicators (GHG emissions, resource consumption, and working conditions) for most prevalent everyday consumption behaviors would be desirable.

3.5.3. Key Lessons Learnt REFOWAS

A Hybrid-LCA is an appropriate approach, when aiming to balance whole sectors, instead of single products. The first step of disaggregating Input-Output-Balances, based on economic total cost accounting (Top-Down) and then specifying the database with additional product LCIs and close data gaps with further statistics (Bottom-Up), results in a more valid, balanced and comprehensive model than using only one of these approaches.

The difficulty of this approach is the comparability between different studies. The steps and data sources have to be fully documented, so that others can understand and follow this approach. This is indispensable for all sustainability assessments. Otherwise, it only leads to further LCA-results, which cannot be explained, and therefore, not compared to each other.

It was shown that many approaches could be used to assess sustainability. However, because there are many degrees of freedom (for example, the question of the emissions factors used, see Section 3.4.2; or the chosen indicators; LCA-methods) and fewer guidelines for LCA-modelling, often the comparability is not given. More transparency (in methodology, databases, choice of indicators) could help to lean on further approaches, and therefore, make them comparable. We suggest a minimum quality report for each comparable study that contains, besides an uncertainty report, definitions of the regional level and system boundaries, as well as the description of indicators. Further, a publication of the steps undertaken to get the results is essential [78].

3.6. Analyses

This subsection is based on the structure of the previous presentations. It analyzes external/ context-related challenges, as well as those depending on the internal assessment logic. As three topics—two mainly external and one mainly internal—emerged as common challenges across all case study contexts, we deemed the following analysis structure as appropriate:

(a)

External/context-related challenges

• Differentiation between deliberation and assessment: First of all, SA actors should be aware whether the primary purpose of SA is a more discursive (maybe political) process, e.g., with the goal of organizational learning, or more analytical in the sense of a scientific assessment. The assessment of sustainability and the deliberation about sustainability as a criterion of assessment follow different purposes. However, both belong to a sustainability assessment.
• Feasibility: After the purpose of an SA has been clarified, and the values made transparent, actors have to deal with implementation-related challenges. SA must be practical in different areas and on various levels, in particular, with regard to data quality, data availability and data reliability, and to the tensions between feasibility and rigor.

(b)

Assessment-internal challenges

3.

Normative basis for assessment: Becoming aware that norms and values affect how a SA is being performed and interpreted is important. In both—deliberative and analytical—cases, the standard of evaluation used to define development as sustainable development is not purely scientific, but rather contextual and transdisciplinary. Here, transparency is important, in order to make the normative foundations of the evaluation traceable and assessable.

3.6.1. External/Context Related Challenges

Deliberation and Assessment

Two levels of SA can be distinguished: On the one hand, SA is a method to assess the sustainability of products and services, aiming to arrive at an estimate of the sustainability performance. On the other hand, SA addresses sustainability as an objective to initiate a discussion on how (best) to assess the sustainability performance of products, aiming to stimulate individual and organizational learning.

The focus of the tool—deliberation or assessment—depends on the purpose of the SA. At the heart of the BiNKA project was an intervention study that focused on the effects of a consumption-specific mindfulness training on participants. The intervention, therefore, specifically targets the consumer mindset and consumer behavior, including the purchase, usage, and disposal of food and clothing, as well as its socio-economic and ecological impacts. This approach differs from NAHGAST, in the sense that, here, intervention targets consumers through the products and their presentation. On the other hand, the objectives and methods of the REFOWAS project involve less than that of intervention: The overall goal of the SA in the REFOWAS project was to analyze the ecologic impacts of avoidable food waste in the value chains of the agriculture and food sector in Germany. Therefore, a consistent and complete model of the German food sector, including food waste, was created, through which the potential of avoiding food waste was analyzed in different case studies. Therefore, REFOWAS addresses the issue of SA more analytically and refers to a quantitative evaluation of impacts. Compared to the approach of SA within NAHGAST and BiNKA, there is less need for deliberation within REFOWAS, as the reduction and avoidance of unnecessary food waste are assumed as being desirable in a value-based debate on sustainability, and the targets—avoiding food waste—and their operationalization is less ambiguous. However, some deliberation is required by setting benchmarks to be set discursively. This discursive setting of benchmarks is also applied in NAHGAST. It also refers to overarching frameworks for sustainability.

Feasibility and Data-related Challenges of the Assessments

The key challenges identified in the case studies mention some common difficulties and challenges that can partially be generalized across contexts. In particular, all three projects face data quality, data reliability and data availability as key challenges.

Especially in the field of nutrition, data quality varies from one to another product. In most cases, data are not as differentiated as they need to be, because they do not reflect different cultivation variants (e.g., organic/conventional, open field/greenhouse, breed or variety, location quality). The differing data quality is caused systemically, since agricultural products cannot be generated under completely controllable conditions (unlike cars, for example), and therefore, the relation between resource input and harvest fluctuates significantly from year to year, depending on, e.g., weather conditions or infestation. In addition, there may be some fluctuation between different accounting institutions for one and the same product. For many products, however, no data is publicly available at all, so ingredients have to be linked to data from similar products.

Besides the challenge of data quality, the problem of manageability and feasibility of the developed tools and methods arises: The NAHGAST tool was designed on a scientific basis, and although a handling for the kitchen staff was considered a priority, the tool remained rather complex, and for some of its features and functioning, an explanation by scientists is necessary. Finally, by carrying out many iterative loops with the practitioners, the NAHGAST team was able to enhance the feasibility of the NAHGAST tool during the project considerably compared to the initial version. Further improvements, especially regarding the inclusion of the tool in the management process of kitchens, is, therefore, eligible. According to this assumption, the opportunity to meet the challenges of the NAHGAST tool and others, is increasing. This can be achieved by creating acceptance and motivation to use these tools and also act upon them—the latter has presented itself as a challenge, especially for REFOWAS.

Another difficulty in SA, regarding reliability, as well as feasibility, are the individual decisions that need to be taken by the evaluator. Inside of parent methods, there are often a lot of suitable subordinate methods, like the choice between an attributional, consequential or cutoff-approach or the choice of the allocation method, functional unit, geographic reference or included processes.

3.6.2. Assessment-Internal Challenges

Normative Assessment Principles

SA contributes to making the values that underlie entrepreneurial action explicit, and hence, debatable, and contributes to the awareness of the practitioners, especially in terms of conflicting goals and interests, and concerning the question: What really is important?

Regarding the normative basis for evaluation, the primary task is to derive measurable criteria from the idea of sustainability. These criteria cannot only be determined objectively, but also have to be based on value judgements: Who is making those judgements? What value judgments will be considered? How transparent are these judgements, and what are the related outcomes?

So far, these questions have not been asked sufficiently in the debate on SA, as well as Horcea-Milcu et al. [79], as one of the most recent publications in the field of SA emphasizes. It points to the importance of understanding that decisions about how to assess sustainability are value-driven. In other words: “[S]ustainability assessment is unavoidably permeated by needs for value-laden choices in the face of uncertainty” [80]. Furthermore, reviews and further development of national sustainability strategies show that values vary over the years and are influenced by external impulses (e.g., the new version of the German Sustainable Development Strategy 2016 that picked up the momentum provided by the 2030 agenda by integrating the SDGs as global reference for national targets) [81].

The selection of indicators taken up in the SA depends on the context, allowing for case-specific interventions and innovations. In NAHGAST, the main criteria for the selection of indicators were: Communicability, Feasibility and Scientific Relevance. The indicators were chosen on this basis raise the problem of compatibility, comparability and integrability. The NAHGAST-tool combines very different indicators; ecological and social indicators are combined with nutritional values, and quantitative and qualitative measures are put into relation. This proved to be a problem in designing the tool consistently, as the question appears whether it is possible to juxtapose ecological, nutritional, social and economic indicators or whether they should remain in different categories—a problem that has both methodological and value-based aspects. The definition of sustainability affects its operationalization. The case studies show the importance of a context-dependent definition. In this regard, the three projects not only show common grounds, but also demonstrate significant differences. The projects differ in their understanding of sustainability: Sustainability within NAHGAST involves four dimensions (ecological, health-relevant, social, economic), while sustainability in the understanding of BiNKA considers two dimensions (ecological, socio-economic). However, both projects have an understanding of sustainability that involves various different indicators. In contrast, REFOWAS has a different starting point as the project focusses on economic and ecological aspects, but especially on the aspect of food waste.

4. Discussion

In Section 3, we compared and analyzed the SAs in the three chosen projects regarding the external and internal features and challenges they have to deal with. In Section 4, we discuss these results against the background of our SA experiences and further scientific literature. It should be noted here that the field of sustainability research in general and SA, in particular, is very diverse. There are established approaches, such as Bellagio STAMP, which are more on the macro level. “The authors of the STAMP had three distinct audiences in mind that could benefit most from applying the Principles in their monitoring and assessment practice: The communities involved in developing alternative metrics systems, the communities focused on integrated assessment and reporting, and those practicing project or policy-focused evaluation.” [5]. So the principles formulated there are relevant to sustainability research, and in particular, to SAs at the level of sustainability policies, but they are of rather limited use at the level of individual projects we have presented, which ask how SA can be put into practice in different contexts on the level of sustainable products, consumption and lifestyles. We consider this level as equally important. As several of the Bellagio STAMP principles seem to match with our results, we do integrate references throughout this discussion section. Other principles (e.g., defining scope and indicators) may address issues that are certainly relevant to the projects we look at, but they are not at the center of our discussion of similar challenges and will, therefore, not be discussed in this Section. Regarding key challenges, such as the clarification of the main purpose of SA—deliberation or assessment—the disclosure of values of those involved, our discussion focuses on other issues than guiding principles as Bellagio STAMP (or goes beyond the aspects mentioned there). Our findings indicate that existing systems and guiding principles suggested for doing SA, such as the Bellagio STAMP principles, are helpful and could benefit from more explicitly requiring transparency on normative value judgements in order to pay justice to broad participation and to enable effective communication.

4.1. External/ Context Related Challenges

In some publications in the field of SA [82,83,84], the question of the main purpose of SA—deliberation or assessment—is discussed. The fact that in numerous others [11,20,85,86,87,88,89,90] this question it is not explicitly stated, seems to be a bit surprising given the assumption that—especially in transdisciplinary intervention research—the participants of a SA do not necessarily have a shared understanding (or maybe even a concrete idea at all) of sustainability. Moreover, in Bellagio STAMP, the postulation “not to lose sight of the purpose of the assessment” is rather a side aspect of principle 2 (essential considerations) that points in the direction of the problem we have raised.

Our three projects are not excluded from the critical finding that the reflection about the main purpose of SA—deliberation or valuation—would deserve more attention in many SA situations, even though deliberation about sustainability and how to operationalize it played a major role for the tool development in our projects. In this sense, the project approaches are also in line with principle 1 (guiding vision) of Bellagio STAMP, which also calls for the development of a common understanding of sustainability among the actors involved [5]. In the cases where SA is not a routine operation in an established setting, it is more than a technical accounting process since the fundamental questions ‘‘sustaining what, for whom and why?” [91] should be discussed with the stakeholders. Nevertheless, there still is some need for methodological innovation to conceptualize and implement deliberation processes in the SA context with stakeholders [83,91]. The necessity and kind of deliberation that frames SA, of course, depends on the context the SA is embedded: “[F]ormal or informal, legally prescribed or voluntary, science-driven or policy-driven” [83]. Irrespective of a more or less strong orientation at higher-level regulatory systems or political decisions, all three examined projects have a rather informal (in the sense of not executing a legal act or similar), voluntary and science-driven focus. This results in a certain indeterminacy which underlines the meaning of deliberation. The demand for a conscientious deliberation process in the three projects is further increased by the fact that the projects leave the paths of scientific routines with their approaches. Therefore, the authors encourage researchers and practitioners “doing” SA to plan adequate time and personal resources and methodological preparation to carry out a thorough deliberation process with the participating stakeholders. For the three presented projects, the deliberation process within the scientific project teams and with stakeholders is an integral part of the project and SA design—even if this aspect was not explicitly addressed in the context of the conception of the projects. In this respect, the approaches of the three projects are in line with Principle 7 (broad participation) of Bellagio STAMP [5]. At this point, the potential for improvement could be brought by an approach that is more guided by theory and more stringently oriented towards methods of systematic and controlled integration of deliberation into the process of SA development. Some literature [83,84,91] with a focus on methods, designed for different SA contexts, may help researchers with this challenge.

In Section 3, data quality, data reliability and data availability have been identified as key challenges in all three project contexts. In this, we conform to many other authors. For example, Guinée lists this problem first [3], Finkbeiner et al. call it a “big challenge” [20], Sala et al. make the important statement that “uncertainties associated to results are barely discussed in literature, whereas, they could be extremely high due to data quality and availability” [92], and the EPA states that “data availability will, in part, determine the necessary tool” [93]. The Bellagio STAMP claim that “data quantity and quality continue to be serious problems” [5].

Wiek et al. mention the “lack of resources for primary data collection” [94], which can be seen as an important reason for the problematic data situation in many funded projects. In the NAHGAST project, the lack of (mainly ecological) data for the NAHGAST calculator indeed led to the leaving out of many ingredients that are used in gastronomy, which was by far the most mentioned criticism by practitioners. Whether this problem can ever be solved, given the high costs of collecting primary data and ever-scarce funding resources, seems to be as uncertain as to the data situation in many projects. On the other hand, the problem of the lack of mutual disclosure of data, which has already been addressed in this article, could more likely be solved within the scientific system by a higher degree of transparency among all those involved, at least where no economic utilization constraints prevent disclosure. In the quest for maximum transparency in terms of data, our results are in line with the corresponding claim of Bellagio STAMP in principles 5 (transparency) and 6 (effective communications) [5]. In addition to the demand for openness and transparency, however, we did not find concrete solutions at this point in the further.

Regarding reliability and feasibility, individual decisions need to be made by the evaluator, which was identified as another challenge. Suh et al. confirm our SA experiences that the “decisions on inclusion or exclusion of processes in analysis (the cutoff criteria) are typically not made on a scientific basis” [82]. They further state, that “the requirement of deciding which processes could be excluded from the inventory can be rather difficult to meet because many excluded processes have often never been assessed by the practitioner, and therefore, their negligibility cannot be guaranteed.” [82] According to Suh et al., the neglect of capital goods can lead to a significant underestimation in LCI.

Clune et al. underline the necessity of common functional units in food LCAs, to simplify the comparison of reports, avoid misrepresentation and strengthen the validity of comparisons [95]. Hunkeler et al. underline the fact that “an LCC practitioner might be confused regarding how to make the most appropriate choices” [96]. Jeroen Guinée [97] emphasizes not least, because of his presentation at the LCA Food 2018 in Bangkok: “LCA: [E]verything is relative, and nothing is certain”. Every SA, as was shown in the three projects and the previously mentioned literature, is always subjectively and individually influenced, which makes the comparison between different studies very complicated.

4.2. Assessment-Internal Challenges

As stated in Section 3, values of the involved actors have an influence on the SA. According to Dijk et al., this problem is inherent in contemporary society: “Lack of consensus on ends or means is directly associated with the plurality in values and interests of today’s society” [83]. The particularly important role that values play in the field of nutrition is shown by Alrøe et al. [88], while the aspect of making actor’s values transparent is considered only indirectly and rather marginally in Bellagio STAMP (principle 5 transparency: “[E]xplain the choices, assumptions and uncertainties determining the results of the assessment”) [5].

All three projects conduct (at least among others) research in the field of nutrition. Everyone is affected by nutrition and their own opinion that is, among others, influenced by habitual aspects in social milieus, cultural, philosophical and ethical backgrounds [98,99], as well as (dis)information in media [100]—the latter mostly about health aspects. Sometimes the topic of nutrition is downright ideological [101]—the discussion of what people are “allowed” to eat is often felt as an encroachment on personal lifestyle [102,103]. Within the three projects, we tried to deal with this challenge by creating a discourse that is based on a solid scientific foundation. However, making values explicit was no part of the SA tool development agenda, which could be a sensible task for further SA projects. This may be all the more relevant when researchers are not familiar with the cultural backgrounds of the participating practitioners, as is not only the case when doing research in emerging and developing countries [104], but also within the ethnically often very diverse teams in the catering sector.

In the context of values, it should be mentioned that, of course, indicators from different dimensions are affected differently by underlying values. Ecological and health-related indicators, on the one hand, can be derived from the current state of research on ecological and health-related questions. This area of research is partially based on normative objectives, such as the 1.5 degree target of the Paris Agreement, the concept of reducing the consumption of resources by a factor of ten [105] or the endeavor to prevent health damage. However, these normative foundations are broadly accepted by the scientific community. The situation in the area of assessment of social indicators is quite different, e.g., for the question: Which extent of ingredients needs to be produced in an acceptable manner, for both humans and animals, for a meal to be sustainable. In this respect, there is a lack of scientifically approved target values, which is why a higher level of subjectivity is included in that context. To balance the subjectivity of the approach, transparent stakeholder dialogues on the social target values should be conducted, as was done in the NAHGAST project.

The discussion about normative assessment principles underlines that in SAs—in all contexts –normativity is an inevitable and integral part actors have to deal with, because they necessarily need to make assumptions and work with heuristics. SA should, therefore, be a joint, stakeholder-based deliberation, with increased transparency—when it comes to normativity.

Therefore, we view SA not only as tools for measuring, but also to encourage scientists, and above all, practitioners getting involved with the topic of sustainability. The obtained knowledge can lead to a change in behavior, for example, in terms of consumption. This shows the interaction between SA and values: One the one hand, SA projects are influenced by the actor’s values and assumptions, on the other hand, the SA could have an effect on the user’s values in turn.

In brief,

Table 5. Similar challenges and solution approaches (source: Own work).

Challenges	Similar Challenges	Solution Approaches
External/Context Related Challenges	The main purpose—deliberation or assessment—has to be worked out; deliberation about case-specific sustainability should definitely be done, although the importance of the deliberation process depends on the ambiguity of the assessment subject. Data quality, data reliability and data availability can be seen as key challenges in all contexts, as well as the feasibility of the SA tool, especially if gastronomic practitioners shall be able to use the tool.	A sufficiently detailed, participative discussion process with all key stakeholders is to be integrated into the project design to receive a common understanding of sustainability. The lack of available data and insufficient resources for creating primary data can hardly be solved in single projects. Researchers should commit to mutual transparency regarding data and methods. Feasibility can be enhanced by a transdisciplinary, participative project design with iterative improvement loops; further technical solutions, e.g., for the integration in the usual software of companies, should be sought.
Internal Challenges	The selection of indicators is not exclusively based on scientific knowledge, but also on normative assessment principles like underlying values—this becomes all the more important, the more the field of research is influenced by, e.g., cultural norms and values, and the more scope for interpretation is left to the subject of the study.	In interdisciplinary teams, researchers should reflect their priorities regarding certain sustainability aspects and SA indicators and integrate key stakeholders in this process.

shows the key findings of Section 3 and Section 4.

5. Conclusions

In this article, a ‘most different case study approach’ was carried out for identifying common challenges researchers are facing in using SA approaches and how to overcome them. We envisaged to emphasizing practice-oriented aspects for the benefit and encouragement of other researchers and practitioners. The analysis indicates that doing SA is important across different contexts, but that it is especially relevant to identify the specific purpose, challenges and additional benefit of doing SA within each context. Stemming from the limitations of our most different approach, the need to conduct further, more in-depth case studies becomes apparent—if possible, in the form of a separate research project, instead of an additional task to the actual research work.

On the basis of the deliberative analysis of the research topic, the three projects created different assessment tools and models that supported context-specific interventions in the respective field of research. Thus, the tools and models fulfilled two functions: Firstly, they carried out the SA of the respective item. Secondly, they served the deliberative discourse among the participating scientists and practitioners on the nature of sustainability, in relation to the respective object of observation. This can motivate actors to deal with content and methodological issues of the subject and lead to individual and organizational learning. Even though this aspect was rather implicit in the projects, the cases indicate that a substantial occupation with the case-specific character of sustainability helps to manage the challenge of gaining a common understanding of the researchers and the stakeholders about sustainability. Researchers should integrate sufficient resources for this deliberation process in their projects. Further research potential, especially regarding methods for designing deliberation processes, can be assumed in this area.

Regarding normative assessment principles, measurable criteria have to be derived from the normative idea of sustainability. This is why sustainability evaluations always have a normative basis and depend on the societies’ and the actors’ values. Those values vary over time and are influenced by external impulses. Indicators from different dimensions are affected by underlying values differently. Some normative (especially ecologic and health-related) foundations are broadly accepted by the scientific community, while the situation in the area of assessment of social indicators is quite different. Hence, the case studies indicate that transparent disclosure of underlying values has been underestimated so far and show the importance of a context-dependent definition of sustainability. The discursive integration of stakeholders in the tool designing process can help to deal with their underlying values and to enhance the assessment’s objectivity. Again, a need for further practice-oriented methods must be emphasized.

Furthermore, although all assessment tools aim to measure sustainability in general and in their specific focus area, they all encounter problems with data reliability and data availability on different (indicator) levels. Therefore, within further research, it is crucial to work on data availability. In order to achieve this, each SA is encouraged to create transparency on which existing data were used and ensure the data availability that was generated through the SA for further research. Data disclosure plays an important role with regard to comparability and validity assessments. Therefore, there should not only be a call for the availability of primary data, but rather more transparency in the publication of the results (such as for methods of syntax, allocation, assessment indicators, as well as database, software, etc.) is required. In that sense, using the same database and applying the same methods can still lead to different results, since different parameters or different weighting of these individual parameters are stored within different software. Only comprehensive transparency regarding the chosen indicators—and the definition of the scope for a feasible and appropriate SA under the circumstances of data availability and time restrictions—can lead to an improved understanding of the sustainable steps taken and the data used. Hence, comparability can be achieved, when everyone avoids hiding data and information needed to retrace the sustainability assessment; the comparison and disclosure of different methods and their problems in terms of triangulation can lead to an overarching and deeper understanding of how to achieve sustainability goals, for specific problems and in general. Nevertheless, our case study confirms that the creation of primary data, especially ecological data, is one of the most important and urgent needs for further research in the field of SA to ensure the validity of SA results.

One reason for the difficulties encountered might be that literature does not provide standardized steps for developing and carrying out SA. The goal is to promote sustainability by taking different views and aspects into account and combining them with manifold and suitable methodologies. As different as the settings and methods might be, the development of different approaches to assessing sustainability leads to useful results in various contexts. Accordingly, SA projects should be encouraged to combine different methods to cover the whole spectrum of sustainability and face the challenge of how to integrate various methods.