Sustainability Experiments in the Agri-Food System: Uncovering the Factors of New Governance and Collaboration Success

Abstract

In recent years, research, society and industry recognize the need to transform the agri-food system towards sustainability. Within this process, sustainability experiments play a crucial role in transforming the structure, culture and practices. In literature, much attention is given to new business models, even if the transformation of conventional firms toward sustainability may offer opportunities to accelerate the transformation. Further acceleration could be achieved through collaboration of multiple actors across the agri-food system, but this calls for a systems approach. Therefore, we developed and applied a new sustainability experiment systems approach (SESA) consisting of an analytical framework that allows a reflective evaluation and cross-case analysis of multi-actor governance networks based on business and learning evaluation criteria. We performed a cross-case analysis of four agri-food sustainability experiments in Flanders to test and validate SESA. Hereby, the key factors of the success of collaboration and its performance were identified at the beginning of a sustainability experiment. Some of the key factors identified were risk sharing and the drivers to participate. We are convinced that these results may be used as an analytical tool for researchers, a tool to support and design new initiatives for policymakers, and a reflective tool for participating actors.

1. Introduction

Research, society and industry all acknowledge the need to transform various socio-technical systems (e.g., energy, mobility, food, and housing) towards more sustainable systems. One important case is the agri-food system due to its high dependency on the availability of natural resources (e.g., soil or fossil fuels), its intensive energy consumption and other socio-demographic and ecological pressures [1,2,3]. Experimentation is an essential element because experiments can initiate change and stimulate social learning [4,5,6,7,8]. Moreover, learning through experimentation can strengthen the adaptive capacity of the agri-food system [9]. Furthermore, collaboration between multiple actors is necessary to develop, apply and establish new innovative ideas and practices, and is linked to the socio-economic and ecological sustainability of the agri-food system [10]. Various definitions and terminology about experiments already exist such as sustainability experiments (e.g., [7]), transition experiments (e.g., [8]), socio-technical experiments (e.g., [11]), governance experiments (e.g., [4]) or niche experiments (e.g., [6]). Despite this variation, all experiments have some common characteristics, e.g., a high risk of failure, being small-scale, local or regional, practicing radical innovation, being practice-based and learning-by-doing [7,8].

Experiments and their role within a transformation process have been thoroughly studied across a wide range of systems such as housing (e.g., [12]), water (e.g., [4]) and urban labs (e.g., [13]). Moreover, agri-food systems research increasingly focuses on alternative food systems such as short supply chains (e.g., [14]), local food initiatives or community based food supply (e.g., [15]). Experiments are a core object within transition management studies [16,17,18,19]. The multi-level perspective is often used to explain transformation processes as an interplay of three analytical levels. At macro level, the landscape is seen as an exogenous environment beyond the influence of niche and regime actors (e.g., deep cultural patterns or global policy measures). Second, at meso level, regimes form the dominant rules and practices such as common beliefs, capabilities, lifestyles and institutional arrangements. Finally, at micro level, niches or new radical innovative practices occur and act as “protected spaces” [16,20,21,22,23].

Although multi-actor sustainability experiments occur, the link between businesses from the regime as well as new emerging firms is often overlooked [1,6,10]. Involving existing businesses offers various opportunities to sustainability experiments for transformation [1,6]. Within the agri-food system, an individual firm (e.g., farm, food manufacturer, and retailer) is commonly part of a supply chain. An individual firm can tackle sustainability challenges individually but by collaborating throughout the chain, it can exploit more opportunities [24,25,26]. Moreover, firms could also get a competitive advantage through early adoption. Organisational relationships provide opportunities by decreasing business uncertainty and sharing of resources. Therefore, collaboration may stimulate or even accelerate a transformation towards sustainability. Moreover, the private sector, and especially retailers, are at the focal point of a transformation due to their growing influence in the global agri-food system [6,13,27,28,29,30].

An experiment involving all supply chain actors (from farmer to consumer) would generate the highest benefits to transform the agri-food system towards sustainability. This is a relatively new type of governance among chain actors combining structural and relational governance elements. The structural governance is the more explicit and formal aspect of the relationships within experiments while the relational governance contains more the normative and informal aspect [31]. How this new governance system influences the collaboration success of such experiments is so far unknown. To study these aspects of governance and the collaboration success, a systems approach should be applied that includes the whole supply chain, the involved actors and their interrelationships.

Frameworks to evaluate sustainability experiments (e.g., [8,32,33]) as well as the supply chain governance and collaboration already exist (e.g., [31,34,35,36]), but are rarely combined. The existing frameworks do indeed not allow cross-case analysis and do not include the following types of evaluation criteria: (i) business-oriented (e.g., transaction cost characteristics); and (ii) learning-oriented (e.g., interaction or drivers to participate) [8,29,37]. Therefore, approaches to assess sustainability experiments using evaluation characteristics that are both business- and learning-oriented would be highly welcome. Furthermore, empirical findings regarding how firms or agri-food supply chain actors within experiments could foster or accelerate a transformation towards sustainability are scarce. Therefore, this paper aims to answer two research questions: (i) Which systems approach allows evaluation of sustainability experiments within the agri-food system? (ii) Which factors influence the collaboration success and related performance of sustainability experiments?

Defining sustainability is challenging and normative; numerous literature sources start from the Brundtland definition “Development that meets the needs of the present generation without compromising the ability of future generations to meet their own needs” but this results in many different implementations and interpretations at practice level [38]. Sustainability is thus a contested and evolving concept with uncertainty about values, interests and methodological approaches due to the difficult cause-effect relationships and interrelationships with other factors [39,40,41,42]. As a consequence, not one possible pathway nor one “sustainable” system state exists [10,43,44]. Therefore, we define a sustainability experiment as “a highly innovative initiative to improve the sustainability state of the whole chain through new arrangements of collaborations”. They consist of self-governing innovative action networks and involve external or horizontal (e.g., research, NGOs, and policymakers) and internal or vertical (e.g., farmer, food manufacturer, and retailer) actors of the agri-food system.

To answer our two research questions, we developed and applied a new approach called the Sustainability Experiment Systems Approach (SESA). This approach is based on institutional economics, transition theory and evaluation methodology. The aim of the new analytical framework is to reflectively evaluate sustainability experiments, with the goal of allowing a cross-case analysis. The aim is to include the agri-food system within the wider socio-ecological system level and to take the organisational and relational structure of experiments into account. To test the framework, we have evaluated and cross analysed four in-depth case studies of sustainability experiments for transformation in the agri-food system in Flanders (the northern part of Belgium). All four experiments are self-governing networks that were initiated through a specific subsidy call. A steering group selected them to receive a small grant consisting of funding and evaluation support. The remainder of this paper is organized as follows: Section 2 describes the sustainability experiment systems approach (SESA). Section 3 contains the case study research. Section 4 presents the empirical results. Section 5 contains a general discussion and conclusion.

2. A Systems Approach to Evaluate Sustainability Experiments

A new sustainability experiment systems approach (SESA) was developed to study innovative sustainability experiments focusing on new arrangements of governance, collaboration and interaction. Several background theories are used to develop SESA, including new institutional economics, transition theory and evaluation methodology. This combination will achieve to analyse experiments using both business and learning criteria. Moreover, SESA does take multiple levels into account such as the firm, the agri-food system and the socio-ecological system level. Furthermore, it is also multidimensional (addressing the inter-organisational set-up, process, outcomes and performance) [45]. Transaction cost economics, which is deeply rooted in new institutional economics, is used to incorporate the business perspective. In general, collaboration networks reduces transaction costs, facilitates cost avoidance, develop a shared vision that create a richer understanding, initiates a learning process and allows small actors to increase their impact on the agri-food system [39,46]. Furthermore, transaction cost economics allows new types of governance to be identified, based on the coordination and control mechanisms. This theory has been proven to be applicable for identifying transactions and governance types of hybrid forms in agri-food supply chain networks [36,47,48]. Hybrids are all organisational forms located between the spot market with immediately completed transactions without a durable relationship between anonymous transacting parties, and hierarchical forms of governance [49,50]. At present, hybrid forms (e.g., relation-based alliances, joint ventures, multi-actor networks, group of producers) are also the most common forms of governance in the agri-food system [47,51,52,53]. Therefore, most business evaluation criteria are derived from transaction cost economics such as transaction characteristics (e.g., product features or uncertainty) and coordination mechanism (e.g., contract type or allocation of decision rights) [54,55].

In addition, the goal of sustainability experiments is to contribute to and accelerate the transformation towards sustainability. Within these experiments, horizontal as well as vertical actors of the agri-food system should be included. This implies that the process, learning effects and performance must also be evaluated. Transition management makes it possible to structure the evaluation of the process and the analysis of the participating actors. It also makes it possible to study the effects of the sustainability experiments within the wider context by evaluating the performance of the experiments. Some evaluation criteria inspired by transition management are interaction and drivers to participate. Evaluation methodology and the evaluation of various participatory initiatives [45,56,57] has been used to structure our analytical framework and to identify criteria useful for evaluating societal effects and the various outcomes. The outcomes are evaluated in terms of collaboration success or failure and future collaborations [58].

This extensive literature review of new institutional economics, transition theory and evaluation methodology revealed numerous possible evaluation criteria. From the resulting list, we selected a set of evaluation criteria based on their relevance, applicability, and accuracy, i.e., how appropriate are the criteria, are the criteria suitable and understandable for analysis and how precise are the criteria to evaluate sustainability experiments. Therefore, closely related criteria were clustered into one, and all criteria are defined. For instance, the criteria “expected mutual benefits” and “social expectations” are clustered to expected social gains. Figure 1 presents the resulting analytical framework. The following paragraphs discuss these dimensions and their corresponding evaluation criteria in depth.

2.1. Dimension 1: Input of Inter-Organisational Set-Up

The inter-organisational set-up is the combination of the structural and relational governance [31].

Table 1. Description of the evaluation criteria of Dimension 1: “Input of inter-organisational set-up”.

Evaluation Criteria	Short Description	Assessment Attributes	Sources
Transaction characteristics	“Transaction characteristics” specifies the structure of the transaction within a sustainability experiment. This criterion consists of product features, transaction cost characteristics, risk sharing and task decomposability.		Banterle and Stranieri, 2013; Claro et al., 2003; Hagedorn, 2005; Hudnurkar et al., 2014; Jang and Olson, 2010; Matopoulos et al., 2007; Menard and Valceschini, 2005; Sauvée, 2013; Williamson, 1997; Williamson, 1998 [34,36,48,51,54,55,64,65,68,71]
Product features	“Product features” is the theme of the innovation such as joint product development, logistics, joint demand management or communication.	Type of product features
Asset specificity—uncertainty—frequency	“Transaction cost characteristics” are determined by asset specificity and specificity of investment, referring to the extent to which non-arrangeable investments are specialized and unique to a task. Uncertainty refers to the (in)ability of actors to have access to all information, and frequency refers to the number of transactions.	Low (−) Medium (±) High (+)
Risk sharing	Risk sharing refers to the distribution of the cost of consequences of a risk among several actors.	Low (−) Medium (±) High (+)
Task decomposability	Task decomposability refers to the possibility to separate the process into distinct tasks.	Low (−) Medium (±) High (+)
Coordination mechanism	The measurement of making actors act jointly and move together towards a shared goal.		Bolwig et al., 2010; Gellynck and Molnár, 2009; Peterson et al., 2001; Raynaud et al., 2005; Sauvée, 2013; Sauvée and Coulibaly, 2008 [36,53,60,61,62,63]
Contract type	The contract type specifies how agreements are formalized based on two elements: is the contract written and is it legally enforceable?	YES/NO
Allocation of decision rights	Who makes the decisions and what is the nature of the decisions?	Centralized/flexible/spread
Intensity of control	Degree of regulation of actors and their corresponding tasks within the network.	Low (−) Medium (±) High (+)
Choice of partners	“Choice of partners” is determined by the selection criteria to choose a partner, the relevance of identity, and the length of relationships.		Claro et al., 2003; Gellynck and Molnár, 2009; Matopoulos et al., 2007; Raynaud et al., 2005; Sauvée, 2013 [34,36,60,62,64]
Selection criteria	The criteria used to choose a partner.	A few/multiple/numerous
Relevance of identity	Do the partners know each other in advance?	YES/NO
Length of relationships	The length of pre-established relationships.	Short (<1 year)/medium (1–5 years)/long (>5 years)
Type of relationships	Type of relationship consists of the level of trust, power distribution, and number of conflicts.		Belaya and Hanf, 2009; Bolwig et al., 2010; Claro et al., 2003; Fischer et al., 2010; Hagedorn, 2005; Hudnurkar et al., 2014; Matopoulos et al., 2007; Menard, 2010; Moschitz et al., 2015; Sauvée, 2013; Thompson and Lockie, 2012; Zhang and Aramyan, 2009 [31,34,36,47,50,61,64,65,67,68,69,70]
Level of trust	The degree to which an actor believes that his/her/its partners are truthful and considerate.	Low (−) Medium (±) High (+)
Power distribution	Do the actors experience power equality or not? What is the effect on the outcomes?	Equal Unequal Diversified
Conflicts	The number of disagreements.	Low (−) Medium (±) High (+)

describes all the evaluation criteria, their assessment attributes and the literature sources. The transaction characteristics are based on asset specificity and specificity of investment, uncertainty and frequency, product features, risk sharing and task decomposability. These evaluation criteria were chosen to describe the structure of the transactions within the network. They are directly linked to the coordination mechanism which contains the contract type, allocation of decision rights and intensity of control. The type of contract allows to distinguish between a written contract or oral agreement and evaluates the legal enforceability of the agreements which aim to decrease opportunistic behaviour of actors [36,53,59,60,61,62,63]. The relational governance is specified by choice of partners and type of relationships. The choice of partners describes the selection criteria, the relevance of identity and the length of pre-existing relationships. These criteria are chosen to study if an ex-ante restriction on the choice of partner occurs and who and how the networks are composed and structured [34,36,60,62,64]. Last, the type of relationships is based on the level of trust, power and conflicts which are included, as these criteria have been identified as key elements of sustainable relationships and collaborations [31,34,36,47,50,61,64,65,66,67,68,69,70].

2.2. Dimension 2: Process

Within a multi-actor network, the evaluation of the process helps to describe the dynamics, co-creation of knowledge and social learning process [39,72]. Therefore, the dimension “process” is described based on the interaction and the drivers to participate.

Table 2. Description of the evaluation criteria of Dimension 2: “Process”.

Evaluation Criteria	Short Description	Assessment Attributes	Sources
Interaction	The interaction is described by frequency of interaction, forms of interaction, information sharing, resource exchange, and the quality of communication.		Claro et al., 2003; Fischer et al., 2010; Hagedorn, 2005; Hudnurkar et al., 2014; Jang and Olson, 2010; Luederitz et al., 2016c; Menard, 2010; Ramanathan and Gunasekaran, 2014; Raynaud et al., 2005 [8,26,47,50,60,64,65,68,71]
Frequency	The number of group meetings.	Meeting/months
Forms of interaction	The meetings that take place such as group meetings. Are meetings arranged by bilateral consultation or on demand or at fixed times?	Type of interactions
Information exchange	The exchange of confidential information between actors through various sources such as face-to-face, telephone or e-mail and the degree of transparency within the process.	Low (−) Medium (±) High (+)
Resource exchange	The process of sharing capabilities, knowledge, assets or investments.	Low (−) Medium (±) High (+)
Quality of communication	The degree to which the shared information is accurate, adequate, reliable, credible, understandable and regularly occurring.	Low (−) Medium (±) High (+)
Drivers to participate	The drivers to participate are described by the expected social gains and the motivation.		Matopoulos et al., 2007; Paavola and Adger, 2005 [34,75]
Expected social gains	The mutual benefits and the initial expectations of the actors at the beginning of the experiment.	Low (−) Medium (±) High (+)
Motivation	The motivation to participate and their commitment during the process.	Low (−) Medium (±) High (+)

describes all evaluation criteria, their assessment attributes and the literature sources. We evaluate interaction based on frequency, information exchange, resource exchange and communication quality. These factors are selected based on previous research focusing on collaboration in supply chains [8,26,47,50,60,64,65,68,71]. Moreover, the drivers to participate are evaluated based on the expected social gains and motivation as these factors have been identified as indispensable in multi-stakeholder processes [73,74].

2.3. Dimension 3: Outcomes

Collaboration is essential to determine the performance of sustainability experiments [10]. Collaboration delivers more satisfied actors with an increased perception of addressing the problem and with perceived enhanced learning effects (individual as well as collective) [42,58,74,76]. Therefore, in this study, the dimension of “outcomes” is described by the collaboration success. Based on other authors [26,58], we define collaboration success as the combination of the perceived social gains, perceived transformative power and perceived learning effects. Furthermore, the outcomes are also described by the future collaborations between the multiple actors of the experiments, either bilateral, in the same, or a different network.

Table 3. Description of the evaluation criteria of Dimension 3: “outcomes”.

Evaluation Criteria	Short Description	Assessment Attributes	Sources
Collaboration success	The combination of perceived social gains, perceived transformative power and perceived learning effects.		Bos et al., 2013; Brown and Vergragt, 2008; Luederitz et al., 2016c; Moschitz et al., 2015; Ramanathan and Gunasekaran, 2014; Schmid et al., 2016 [8,12,26,58,70,77]
Perceived social gains	The perceived individual and mutual benefits of participating.	Low (−) Medium (±) High (+)
Perceived transformative power	The contribution of the experiment to the sustainability transformation.	Low (−) Medium (±) High (+)
Perceived learning effects	The perceived outcomes of the process of knowledge co-creation.	Low (−) Medium (±) High (+)
Future collaborations	Will the collaboration continue?	YES/NO	Schmid et al., 2016 [58]

describes the evaluation criteria, their assessment attributes and the literature sources.

2.4. Dimension 4: Performance

As the goal of the sustainability experiments is to accelerate and contribute to the sustainability transformation of the agri-food system, the performance is defined based on deepening, broadening and scaling-up of the experiments based on the governance analysis of transition management instead of their economic performance [18,19,78].

Table 4. Description of the evaluation criteria of Dimension 4: “Performance”.

Evaluation Criteria	Short Description	Assessment Attribute	Sources
Deepening	The social learning process about new norms and values (e.g., shift in culture, practices or structure) to perform societal functions within a specific context. The result is the introduction of new ideas (culture, practices or structure) in an innovative way.	Low (−) Medium (±) High (+)	Kemp et al., 1998; Porter et al., 2015; Schot and Geels, 2008; van den Bosch and Rotmans, 2008 [29,78,79,80]
Broadening	Broadening is repeating the experiment in a different context and is based on the idea that various experiments with the same characteristics exist simultaneously and can learn from each other over time to become an emerging community.	Low (−) Medium (±) High (+)	Porter et al., 2015; van den Bosch and Rotmans, 2008 [29,78]
Scaling-up	The embeddedness of an experiment in the dominant ways of thinking (culture), doing (practice) and organizing (structure) at the level of the socio-ecological system.	Low (−) Medium (±) High (+)	Porter et al., 2015; Rotmans and Loorbach, 2008; van den Bosch and Rotmans, 2008 [18,29,78]

describes the evaluation criteria, their assessment attributes and the literature sources. Performance is described from the actors’ perspectives, as the effective impact on the sustainability of the agri-food system is not measured. Therefore, other environmental factors such as parallel developments are not taken into account. Besides scaling-up, the opportunities for scaling-up are also evaluated to identify possible barriers and leverages.

3. Case Study Research

3.1. SESA Implementation

We developed an implementation methodology to apply SESA into practice mostly based on reflexive monitoring in action [81] which had already proven its applicability in systems approaches [82,83]. This is an action based monitoring methodology which aims to stimulate reflection of the actors about the project goal, the practices, their organisational structure and the developments to realise their ambition and system innovation. Monitoring is fully part of the process instead of a separate section. Therefore, project goals and ambitions are iteratively adjusted and organisational changes occur without knowing the end state of the project [81]. Our resulting methodology was a three-step methodology stimulating reflective evaluation and allowing cross-case analysis (Figure 2).

The first step encompassed the selection of case studies. In general, case studies can be selected based on different criteria depending on the research question at stake. For instance, case studies in different regions are more difficult (but not impossible) to compare due to contextual factors. Some possible selection criteria are region, size of actor network, innovation phase (i.e., spreading, diffusion, adaptation and transformation), innovation scale (e.g., novelty, niche development or breakthrough and implementation) or goal-orientation. In our study, four case studies were selected. Selection was based on the following criteria: (i) region is Flanders; (ii) transformative power; (iii) sustainability as starting point; and (iv) comprising a whole supply chain and its actors. All the sustainability experiments were policy supported and a learning process with evaluation and reflection was organized starting at the initial phase up to two years. The goal of the evaluation was twofold: to test and validate SESA and to identify the key factors of the collaboration success and its related performance of highly new sustainability initiatives. In our study, all cases had the same innovation scale and phase, the scale was novelty and the phase was adaptation and transformation.

The second step was the in-depth case study analysis with the self-governing action networks as unit of analysis [84]. This analysis was based on frequent reflective evaluations focused on the actors’ perspectives, as they are the best positioned to evaluate the type and degree of outcomes. Moreover, their perspectives could reduce the science and process-centred bias [58]. Three types of evaluation methodology were used. First, two reflective evaluations with the coordinator, initiator or facilitator of the experiment took place: one at the beginning of the process (0–6 months) and one after one year. This reflective evaluation was done with semi-structured interviews based on the evaluation criteria of the SESA framework. The first evaluation focused mainly on idea development, organisational structure and choice of partners. The second focused more on the process and collaboration quality. After these evaluations, a timeline and eye-opener workshop were organized. The timeline workshop aimed to explicitly identify the challenges, successes and learning experiences within a certain experiment, while the eye-opener workshop aimed to present the details of key moments and results of the experiment to outsiders and to organize a joint reflection with various perspectives. These workshops were held after 18 months. The actors of the experiments came together with other external actors and shared and transferred their learning experiences [81]. Subsequently, a third reflective evaluation, which involved all key actors, was done at the end of the evaluation period. This reflection focused on the entire SESA analytical framework and future perspectives. The data were collected through semi-structured interviews. Last, internal reports and documents were collected.

After the first two years of the experiments, the in-depth case study analysis of the different cases was performed as follows. All of the interviews were transcribed and the method of open and axial coding was used [85], with the aid of the software “NVIVO 11”. This method abstracts all information into more manageable pieces of data based on the evaluation criteria and the dimensions of the framework. Figure 3 illustrates a single in-depth case study analysis with the method of open and axial coding. In total, 25 semi-structured interviews were held: four interviews from Case A, nine interviews from Case B, five interviews from Case C and seven interviews from Case D. As a result, for every case study, we described all the evaluation criteria and gave them assessment attributes such as high, medium or low. Thereafter, we evaluated which characteristic influenced which outcome. Importantly, interrelationships could occur: for example, the evaluation criteria level of trust can be assessed ranging from high to low but can influence various outcomes such as future collaborations, perceived social gains or perceived learning effects. These interrelationships are also taken into account as specified by the actors.

The third and last step was the cross-case analysis. To generalize the results, a cross-case analysis was necessary [86]. All the cases were compared and general results were described which can help to reduce the case-specific and context specific results. More specifically, all assessment attributes of the four cases were collected and compared and more general key factors were identified and discussed. The key factors are identified based on their role within the in-depth case study analysis and the difference between “successful” and “failed” sustainability experiments.

During the whole process, different forms of triangulation were used to validate the results [87,88]. Data triangulation was performed by using data derived from different sources such as actors, documents and observations. Moreover, different methods to collect and analyse data were used to ensure methodological triangulation (e.g., scientific and popularized literature, interviews, documents and observations).

3.2. Short Description of Case Studies

All cases are part of the Flemish agri-food system. Flanders, the northern part of Belgium, is a small, highly urbanized region with a high population density. Similar to the agri-food systems in many other industrialized countries, the Flemish agri-food system is exposed to pressures such as an aging population, supply chain concentration, environmental challenges and an increasing focus on a more sustainable food system [89]. Most farming systems are highly specialized (88% of the farms) and are intensified, characterized for example by indoor production systems for pigs, dairy cattle and chickens [24]. Similar to other Western European countries, farms are usually family businesses with individual ownership [90]. The Flemish agri-food system is export driven and characterized by small and medium-sized enterprises [89].

Case A: Valorisation of Organic Surplus into a New Marketable Product

The first experiment aimed to reduce food waste of fresh vegetables during peak production of organic agriculture. To reach this goal, the development of a new innovative platform was necessary to produce a marketable product of second choice vegetables, i.e., vegetables that differ from the norm based on size, colour or structure. After a survey with farmers, courgette was selected because overproduction occurs every year. Product development was courgette pesto. At the same time, they aimed to develop a smart logistics system to optimize the transport of the product from farmer to processor to distributor. For the processing, they collaborated with a sheltered work environment, created for disadvantaged persons who cannot participate the regular job market. Therefore, their sustainability goal was threefold: (i) to reduce food waste; (ii) to make an economic viable product; and (iii) social equity. The involved actors were a coordinator from the organic farmers’ association, 10 organic farmers, a processor with social employment, an organic distributor, a logistics manager, a research institute and a certifier.

Case B: Production, Processing and Consumption of Locally-Grown Soybeans for Food and Feed

The second case aimed to produce, process and test consumption of locally produced soybeans in the agri-food system for food and feed. More specifically, in the human food chain, they aimed to produce soy milk and soy yoghurt with locally produced soybeans without decreasing the quality and nutritional value. In the feed chain, they aimed to produce pigs fed by locally produced soybeans without decreasing the meat quality, taste and nutritional value. This experiment was the first initiative to produce soybeans in Flanders outside controlled field experiments. Their sustainability goal was threefold: (i) to reduce import dependency; (ii) to generate a fair income for the producers; and (iii) to generate an economic viable product. Ten actors were involved, all actors of the human and feed supply chain, i.e., a feed producer, food producer, seeds processor, food processor, feed processor, pig processor, distributor, and a coordinator and a secretary, and two research institutes.

Case C: Shop, Pick, Drive and Deliver

The third case is the experiment called “shop, pick, drive and deliver” which aimed to start a sustainable business-to-consumer distribution platform to deliver fresh and processed farm products to the homes of care-dependent consumers. The origin of this experiment were two regional existing initiatives, i.e., pick, drive and deliver of a firm that aims to deliver services to the homes of care-dependent consumers and shop and deliver, a business-to-business platform between local farmers and catering. More specifically, this experiment aimed to realize a regional smart food hub in an urbanized area to deliver agricultural products and to further upscale in other regions based on the lessons learned from this process. Their sustainability goal was twofold: (i) give care-dependent consumers a larger choice of local products; and (ii) develop a viable logistic system. The network involved seven core actors: a coordinator, a field coordinator, a logistics manager, an information manager, a producer’s organisation, a business-to-business distributor, a business-to-consumer distributor and three external actors (a food safety advisor, a policy actor, and a legislative advisor and policy actor to advise on food safety, legislation and practical implications).

Case D: Socially Sustainable Catering at a Hospital

The last experiment aimed to transform one hospital catering system towards sustainability. To achieve this transformation, a collaboration was set up between the agri-food chains involved and between the agri-food chain and the hospital. Moreover, a learning process was initiated to gain insight within the sustainability principles of socially responsible catering and to create a common understanding. Previous examples have been reported but they all involved small institutions. Therefore, a second goal was to roll out this experiment as best practice and to provide guidelines to easily broaden to other institutions of socially responsible catering (e.g., schools, healthcare organisations or public organisations) and to formulate policy measures to stimulate the sustainability principles of socially responsible catering. The sustainability goal was threefold: (i) ton increase the local visibility of producers; (ii) to make the food preparation and consumption more sustainable; and (iii) to stimulate consumers to make more healthy choices. The involved actors were the hospital and its suppliers, an NGO, a large farmers’ organisation and an agricultural cooperative.

4. Results

For each dimension of the SESA, we describe the results and identify the key factors of collaboration success and performance. At the end of the two-year evaluation process, Cases A and C decided to abort the experiment (“failed”) and Case B and D decided to continue (“succeeded”).

4.1. Dimension 3: Outcomes

As the outcomes are mostly influenced by the other dimensions “Inter-organisational set-up” and “process” and were clearly linked to the performance, we first describe the outcomes. The summary of the results is represented in

Table 5. The assessment attributes of the evaluation criteria of Dimension 3: “Outcomes”.

Outcomes	Case A	Case B	Case C	Case D
Perceived social gains	−	+	−	+
Perceived transformative power	−	+	+	+
Perceived learning effects	±	+	±	+
The collaboration success	−	+	±	+
Future of collaboration	No	Yes	No	Yes

+: High; ±: Medium; −: Low.

.

The collaboration success is the combination of the perceived social gains, perceived transformative power and perceived learning effects. First, actors of Cases A and C identified only a few social gains from the collaboration, such as experience with chain-wide collaboration and new relationships within the agri-food system. In contrast, actors of Cases B and D identified multiple and often overlapping social gains. Some identified perceived social gains are: (i) experience with chain-wide collaboration; (ii) building up a trusted network; (iii) establishment or strengthening of relationships between actors and their organisations; (iv) co-creation of knowledge and expertise; (v) motivation to continue the process; and (vi) more realistic expectations for future collaborations. Second, the actors of Case A perceived no transformative power but stated that the idea and product development of courgette pesto with second-choice courgettes could be further developed and also has potential in a conventional agri-food system. In contrast, the actors of Cases B, C and D perceived transformative power, mostly due to the chain-wide collaboration which increased the impact on the output and practice and increased negotiation power with policymakers. Cases B and C successfully changed the legislation on the use of pesticides for soybeans and food safety for short supply chain initiatives, respectively. Moreover, Case B is lobbying further for subsidies for new crops. The coordinator and facilitator of Case D experienced a large sphere of influence due to the power of the hospital as an example of best practice in socially sustainable catering. In general, almost all actors of all cases perceived that the experiment contributed towards at least some sustainability issues of the agri-food system.

Furthermore, all cases experienced social learning and various learning effects. Even the aborted cases reported important learning effects. Some frequently identified lessons learned across all cases were: (i) actors of supply chains with different visions broadened their view; (ii) despite having different perspectives, if all actors have the same goal, collaboration can be successful; (iii) chain-wide collaboration is expedient for problem solving and increases the impact of the experiment on the agri-food system; (iv) collaboration requires open and transparent information exchange and trust; (v) actors must be highly motivated to continue the experiment; (vi) creating a common language and understanding at the beginning of process is very important; (vii) expectations should be identified and harmonized at the beginning of the process; and (viii) having a motivated and engaged coordinator is important to facilitate process and motivate other actors. Cases A and C stopped collaborating, while Cases B and D continued. Case B did split into two networks (food and feed) but they continue to share information regularly. The overall collaboration success result is low for Case A, medium for Case C and high for Cases B and D. This is clearly linked to the aborted and successful cases. This result is useful when describing the following dimensions with the aim of identifying the factors that influenced this collaboration success.

4.2. Dimension 1: Inter-Organisational Set-Up

The summary of results is represented in

Table 6. The assessment attributes of the evaluation criteria of Dimension 1: “Inter-organisational set-up”.

Transaction Characteristics	Case A	Case B	Case C	Case D
Asset specificity **	+	−	+	n.a.
Frequency	n.a.	n.a.	n.a.	n.a.
Uncertainty	+	+	+	+
Risk sharing **	−	+	−	±
Task decomposability	+	+	+	+
Coordination mechanism
Contract type	no	no	no	no
Allocation of decision rights **	Centralised	Spread	Centralised	Spread
Intensity of control **	−	±	−	±
Choice of partners
Relevance of identity	−	+	±	±
Length of existing relationships	n.a.	LT (>5 years)	LT (>5 years)	MT (1-5 years)
Type of relationships
Level of trust **	−	+	±	+
Power distribution **	Unequal	Equal	Unequal	equal
Amount of conflicts	±	±	±	−

+: High; ±: Medium; −: Low; n.a.: Not applicable, LT: long term, MT: midterm; **: Key factors.

.

Within the transaction characteristics, the product features are specified. Cases A and B developed new products (courgette pesto in Case A and new animal feed and soy milk in Case B). Cases C and D focused on a new logistics system (e.g., direct home delivery) and a new type of communication, respectively. All cases tested new types of collaborations, governance structures and interaction. Uncertainty is high for all cases and the frequency of transaction is not applicable as the phase of innovation was novelty and product and idea development were still in process. A key factor is the asset specificity in combination with the intensity of control. Cases A and C have a high asset specificity and actors made specific investments for the experiment. For instance, in Case A, specific investments such as acquiring new machines or changing the production process to gain an organic certificate were needed to start the test production of the pesto. Furthermore, in Case C, new cooling boxes were developed to preserve the food safety during transportation of the products. High asset specificity and mutual investment creates a higher mutual dependence which increases the risks of opportunistic behaviour and requires higher intensity of control [91]. However, our cases are newly emerging networks without any type of contract and a low intensity of control. For instance, the coordinator of Case C said that control was not necessary as the collaboration is based on trust and all actors know what they are expected to do. In all cases, most agreements were made orally or written in reports but no legally enforceable contracts were drawn up. Because control was limited in Cases A and C, the risk of opportunistic behaviour increased and did arise in Cases A and C as one actor in each experiment started to make unilateral decisions [91]. Little control was present in Cases B and D, but follow-up meetings occurred regularly in both groups. For example, the coordinator of Case B stated that follow up between the group meetings was not needed because every two months, the group reviewed all of the tasks completed.

In our analysis, collaborations that shared risks enjoyed higher success rates. Case B identified possible risks, problems and challenges and formulated possible solutions with all actors of the network in advance. For instance, the research institute highlighted the lack of pesticide legislation at an initial meeting. As a result, the coordinator, research institute and farmers’ organisation immediately took action to acquire a legislative exception favourable to their case. This initial list of possible barriers enabled all actors to perform their tasks even in cases of high risk such as producing a new crop. By drafting an agreement made by producers and other actors of the network stating to finance the yield per ha instead of per ton, the producers were confident enough to take the risk. In Case D, possible risks were discussed in advance but those risks were lower in comparison to Case B. In Cases A and C, risk sharing was low. In Case A, the processor had the highest risk in the form of producing the pesto and becoming organically certified. The processor felt no support by the other actors and thus decided not to take the risk as the costs were too high. In Case C, most of the risks were carried by one person, the business-to-business distributor. Therefore, in our analysis, the risk-reward ratio was too high in Cases A and C, leading to failure lack of achieving the collaborative goal of sharing risks as well as rewards [31,91].

The last evaluation criterion of the transaction characteristics is task decomposability, which was high for all cases. One minor difference was observed: the process of dividing the tasks. In Cases B, C and D, task decomposability was discussed before the start of the experiment at an initial meeting. As a result, all actors knew exactly what their tasks would be as well as the expectations from the other actors. In contrast, in Case A, no initial agreement was made and confusion arose during the experiment.

Partner selection criterion differed widely among the four cases and is identified as a key factor to collaborate successfully. In Case A, partners were chosen based on knowledge and expertise only. In Case C, criteria were knowledge, expertise and power or influence in the supply chain. In contrast, Cases B and D had a long list of selection criteria to choose partners such as knowledge and expertise, power in the supply chain, trust and previous collaborations, goal-orientation and motivation, and degree of importance of sustainability in the organisation. In all cases except Case A, the coordinator knew most of the actors from previous collaborations. Pre-existing relationships are identified as a key factor of successful collaboration. The existing relationships were mostly longer than five years. Moreover, multiple actors had already previously collaborated and stated this previous collaboration accelerated the initial phase to create a common understanding and identify a shared goal of the project. In Case C, one partner was chosen based on power in the supply chain. However, this actor made the unilateral decision (without consulting the coordinator) to stop the experiment, as the test results were not good enough. Most actors also stated that collaboration with unknown partners delays a process as trust should be build up and creating a shared language requires time.

Our analysis revealed trust to be a key factor of successful collaboration. When mutual trust was low, the collaboration success was low, which is observed in Cases A and C. The level of trust in Case A was low as none of the partners knew each other and they all had a wait-and-see attitude. In Case C, at the beginning of the experiment, the level of trust was rather high but diminished during the process as distribution agreements were not adhered to and decisions were made unilaterally. In Cases B and D, the level of trust was high from the initial phase of the experiment due to previous relationships and an identified shared goal during the process. Our results show no difference in number of conflicts linked to the level of trust. In contrast, the level of trust is highly linked to the equality of power distribution, i.e., the more equal the power distribution, the higher the trust. Furthermore, our results show that the power distribution is directly linked to the allocation of decision rights. In Cases A and C, the decisions were allocated in a centralized way, i.e., to the most powerful actor. Consequently, this actor made unilateral decisions during the process which decreased the level of trust and other actors felt unheard and were unsatisfied with the collaboration. In contrast, in Cases B and D, all decisions were equally allocated among all actors. Decisions were made jointly and actors felt no power asymmetry. These cases reveal that the presence of trust between the various partners, an equal power distribution, and a well-distributed allocation of decision rights positively influence the collaboration success.

4.3. Dimension 2: Process

The dimension process results are summarized in

Table 7. The assessment attributes of the evaluation criteria of Dimension 2: “Process”.

Interaction	Case A	Case B	Case C	Case D
Frequency **	1/6 months	1/3 months	1/3.5 months	1/2.5 months
Information exchange **	−	+	−	±
Resource exchange	−	+	+	−
Communication quality **	−	+	−	+
Drivers to participate
Expected social gains **	Conflicting	Homogenous	Low	Homogenous
Motivation **	−	+	+ → −	+

+: High; ±: Medium; −: Low; **: key factors.

.

Interaction is determined by the frequency of group meetings, information and resource exchange and communication quality. Among the cases, frequency of group meetings was only slightly dissimilar. Case D had the highest frequency of group meetings, closely followed by Case B. Information exchange and communication quality did differ greatly between cases: Cases A and C were similar, as opposed to Cases B and D. In Case B, the production of soybeans in feed and food, stakeholders met every three months and the attendance rate was high; even the retailer which was only involved during the last steps of the experiment was present during most meetings. Moreover, information was shared in a transparent way; intermediate results were shared to all actors and a report of every meeting was sent to all partners. All actors also stated that sharing confidential information was no problem within the network. Internal communication was frequent: questions were immediately asked through e-mail or telephone and all actors felt involved during the whole process. The shared information was also evaluated as accurate, adequate, reliable, credible and understandable to all actors. We observed a similar process of information exchange in Case D. In contrast, Cases A and D exchanged almost no additional resources while Cases B and C exchanged multiple resources such as expertise and knowledge, time, existing networks and technical assets. However, the influence of this factor is less observed in our case studies in comparison to the other three factors.

Case A met only every six months and had low attendance rates. At the end of the experiment, no group meeting occurred and e-mail was the only communication mode. Between the meetings, the coordinator communicated only bilaterally without involving all of the actors and leaving many actors without current information. Remarkably, at the end of the experiment, some actors had no knowledge that the experiment had been stopped. The coordinator also stated that the actors were not willing to easily share their information and they needed to be convinced to share their output. For example, to close the financial report it took three months before costs and specific investments were disclosed. The information exchange is therefore evaluated as non-transparent. In Case C, meetings occurred every 3.5 months with low attendance. None of the meetings had full attendance, which created a gap between the actors in practice and the other actors who developed the idea but were absent on the field. Information was shared transparently but less frequently in comparison to Cases B and D and the quality of communication was low: information was sometimes inaccurate and communication was sometimes delayed. Therefore, interaction, i.e., frequency of group meetings, information exchange and communication, positively influence the collaboration success and was experienced as a key factor in our cross-case analysis.

The homogeneity and level of expectations regarding social gains seem to count the most. In Case A, the expected social gains were not harmonized due to their different perspectives. For example, the distributor aimed to have small portion of courgette pesto with a shelf life of at least one year with a high emphasis on the product quality. In contrast, the shelf life of the product was insignificant for the processor as the local character of the pesto was more important and the processor aimed to gain expertise and to create new collaborations. On the other hand, the expected social gains of Case B and D were numerous and homogenous. Some common expected social gains were to gain expertise and to co-create practice-based knowledge, to develop a new local supply chain, to activate and encourage policy makers, and to develop long term relationships within the supply chain. In Case C, the expected social gains were low as not all actors believed in the idea and the potential benefits of the process, e.g., the business-to-business distributor and farmers remained sceptical about the financial feasibility of the experiment from the beginning.

Most actors in Case A had low motivation throughout the process: they stated that they did not know what to expect and gave the experiment a low priority. Although all actors aimed to develop a new sustainable product, their vision on the development process differed, resulting in conflicts. No initial meeting took place to create a shared vision or understanding. In Case C, the motivation was initially high but dropped quickly after the start of the experiment as unforeseen challenges and problems arose such as technical difficulties and communication issues. In Cases B and D, the motivation was initially high and stayed high during the whole experiment. Most actors of Case B even stated that their motivation even increased during the process and their main motivation was the shared vision on the experiment and their goal of sustainability. For example, the seed processor stated that understanding the farmers’ vision really broadened its view, and the distributor stated that most sustainability issues are tackled within one step of the supply chain (e.g., farming, industry or retail) while a chain-wide collaboration has more potential to be effective. Our results show an influence between expected social gains, high level of motivation and the collaboration success of new sustainability experiments. A positive and constructive multi-stakeholder process increased the collaboration success.

4.4. Dimension 4: Performance

The last dimension (“Performance”) studies how the sustainability experiment contributes to the wider transformation process at agri-food system level. The summary of the results is represented in

Table 8. The assessment attributes of the evaluation criteria of Dimension 4: “Performance”.

Performance	Case A	Case B	Case C	Case D
Deepening	−	+	±	+
Broadening	−	±	+	+
Opportunities for scaling up	−	±	−	+

+: High; ±: Medium; −: Low.

.

The experiments were evaluated during a time span of two years, making it impossible to study the effective impact. The performance is studied based on the actors’ perspectives and the short term effects in structure, culture and practices of the socio-ecological system. Actors of all cases perceived multiple learning effects within the chain-wide collaboration network and shared their experiences during a timeline and eye-opener workshop. Social learning and collective learning occurred in all cases to some extent. The participants mentioned that mainly the interaction between the actors and the broadening of their views due to their different perspectives helped to initiate this social learning process and to co-create knowledge. In Case B, the learning effects were given great importance and all actors agreed upon the added value of the experiment and the importance of continuing to collaborate and further experiment and learn. They reported that the experiment and especially the chain-wide collaboration changed their way of thinking and practice. Most actors also stated that although it was the first time they collaborated within a new self-governing chain-wide action experiment, it was essential to continue this approach in the future. A similar process was observed in Case D in the hospital. Individual as well as collective learning occurred. Moreover, all actors perceived strong learning effects due to the various actors that participated and the co-creation of knowledge during the process. However, the coordinator specified that the goal was to learn more evolutionarily and step by step, rather than making revolutionary leaps. Their goal was therefore achieved and deepening took place. Furthermore, in Cases B and D, some actors even changed their view on chain-wide collaboration and sustainability issues. In contrast, in Case A, although some learning effects were present, not all actors made the step towards more critical thinking about the collaboration, governance and interaction of the initiative. In Case C, a social learning process was initiated and did occur, but was stopped abruptly after one year, which negatively affected the learning effects.

It is difficult to analyse if broadening took place based on these experiments, owing to the short time span of the experiments and other contextual factors that influence broadening. The eye-opener workshop enabled the actors to learn from the other case studies and from external actors. In Case A, hardly any broadening took place. Although other experiments arose in organic and conventional agri-food systems to reduce the peak production and develop new products of second choice vegetables, actors of Case A put no effort in diffusing their idea of courgette pesto and lessons learned. In Case B, some broadening could be detected as new networks arose to test new crops at industrial scale when the field experiments are not finished yet and farmers are involved from the initial phase. Moreover, most actors stated a desire to continue to participate in multi-actor networks and sustainability experiments to test new ideas. In Cases C and D, actors reported a high broadening scale as the cases are repeated in different context. For instance, local food home delivery is currently being tested in various regions with various logistical systems (e.g., bike transport, electric cars or food hubs) and Case D is repeated in other social catering institutes. However, the causal link between Case C and the new local food home delivery initiatives is tenuous, although the coordinator is trying to disseminate the idea throughout a large network. In Case D, the results are communicated as a best practice at different locations and the coordinator provided guidelines for other institutions aiming to improve the sustainability of their catering. Recently-emerged initiatives in socially sustainable catering also collaborate with the organisation of the facilitator to transform towards sustainability.

In this research, scaling up was evaluated in terms of opportunities to scale up, because effective scaling up is difficult to achieve in such a short period of time. Case A showed hardly any scaling up opportunities as the experiment to market second choice courgette failed and new production methods and logistical system should be developed and explored before scaling-up is even possible. Case B identified up-scaling opportunities for the production of soybeans and experienced that the involvement of conventional businesses increased their influence sphere within the agri-food system and external communication impact. Moreover, their opportunities even increased as the legislation changed. One year after the experiment, the area of soybean cultivation increased tenfold. The actors also see further opportunities in a time span of five years when knowledge about soybean production is more advanced. In Case C, no scaling-up is possible. Although the idea can be spread, the test results of this distribution experiment are too negative and will not be explored further. Lastly, in Case D, scaling-up continues in the sense that the experiment just started and the hospital wants to continually improve their social catering towards more sustainability. Therefore, the impact and changes of structure, practice and culture have opportunities for expansion.

5. Discussion

In this section, we discuss the key factors of collaboration success and the strengths and limitations of SESA as a systems approach to evaluate sustainability experiments. SESA is discussed with regards to its analytical framework and the empirical results of the cross-case analysis.

The cross-case analysis revealed 13 influencing factors for successful collaboration within newly-initiated sustainability experiments for transformation. Most of these factors are interrelated. Trust and motivation are two central key factors for collaboration success. Although most key factors are interrelated, some factors are directly interrelated while others are more indirectly interrelated. For instance, asset specificity and intensity of control are directly interrelated. The higher the asset specificity the higher the intensity of control should be. In addition, the relevance of identity is interrelated to the length of the relationship, i.e., in most of our cases, if the relevance of identity matters, the length of relationship is often long (>5 years). Furthermore, trust is linked to equal power distribution and a decentralized allocation of decision rights. Last, frequency of interaction is interrelated with information exchange and quality of communication.

Our identified factors are mostly in accordance with previous research from conventional agri-food systems, participatory and transdisciplinary approaches and transition literature. However, no research identified these factors all together. For instance, our results showed that a low asset specificity or a high asset specificity with control are required to initiate sustainability experiments, which is also observed in conventional agri-food chains [51,91]. Second, we observed risk sharing and specific risk agreements at the beginning of a new experiment as a critical factor. Other research identify risk sharing as a central motivation to collaborate as it decreases the uncertainty and specify that the risk reward ratio should be beneficial for all actors [31,91]. Third, we observed that the identity of partners and the length of relationships are crucial to select partners, which is similar to findings of previous studies in conventional European agri-food chains (e.g., [50,52,92]) who indicate that these factors are important to determine the governance type. Fourth, our results indicate a strong link between trust, power distribution and allocation of decision rights. Multiple researchers also state that mutual trust plays a key role in sustainable relationships [26,31,36,64,91,93] and identify power equality as an important factor that influences the collaboration success [31,50,67,92]. The presence of trust between the various partners, an equal power distribution and a diversified allocation of decision making rights could help to successfully initiate new experiments. Furthermore, our results revealed that information exchange, transparency, and effective communication in terms of quality and frequency are key factors for a successful collaboration and for sustainable relationships, which is confirmed by other researchers [25,26,47,50,64,65,92]. Last, expected social gains and a high motivation influences the collaboration success, which is also observed in participatory processes by other researchers [8,26,93,94,95,96]. Expected social gains are even identified as one of the most essential factors of collaboration [26]. More specifically, the motivation should exceed the monetary or reputational benefits, is strongly linked to commitment and is necessary to change the behaviour of actors in practice [8,26,95]. In general, a positive and constructive multi-stakeholder process could increase the collaboration success of new sustainability experiments. These identified key factors could be validated and improved by implementing SESA into more case studies. Moreover, the resulting factors could help policy makers to design and support new initiatives, help businesses to create new partnerships or help researchers and practitioners to design new sustainability experiments with a possible increased chance of collaboration success.

SESA allows multilevel and -dimensional analyses involving the complexity of sustainability experiments within the agri-food and socio-ecological system level. We analysed multi-actor processes with new arrangements concerning governance, interaction and collaboration. The identification of governance types by conventional agri-food chains is also done by other researchers to explain certain trends and observations [7,36,91,94]. However, their focus is mostly economic while our starting point is sustainability focussing on multiple issues such as socio-economic and ecological. Our evaluated experiments were trust-based multi-actor self-governing networks.

Moreover, SESA allows a dynamic monitoring approach. Our implementation approach of SESA allowed a dynamic evaluation of sustainability experiments, with five evaluations held over a time span of two years. Semi-structured interviews were held and workshops were organized which improves the reflective evaluation and formulation of recommendations for decision making [9]. The performance analysed the contribution to the sustainability transformation of the agri-food system and wider socio-ecological system so that other domains (such as housing, mobility, and energy) could learn from sustainability experiments in the agri-food system.

Furthermore, SESA succeeded to combine business and learning criteria. The influencing factors of successful collaboration present both the business and learning perspective which was not studied before. However, our empirical results are based on four cases similar in terms of innovation phase and scale. In our opinion, SESA could potentially be applied in all phases and scales of sustainability initiatives within a transformation process. In our empirical results, the business criteria were less prominent in the evaluation of our cases as contracts were absent and coordination mechanisms were underdeveloped due to the early phase of the experiments. Future research could implement SESA in more mature sustainability initiatives to validate the identified key factors of collaboration success. For instance, sustainability initiatives existing more than five years could be evaluated. Thereafter, the key factors of collaboration success at the initial and more mature phase could be compared. Within the more mature cases, we expect the business criteria to have a greater influence and a more prominent role.

The participatory evaluation approach was time consuming, which is also observed by other researchers [97,98]. Therefore, we recommend future research to study the development and implementation of a self-reflective evaluation tool based on SESA and the key factors. This tool could improve SESA and make it more applicable to sustainability initiatives in various sectors.

In our empirical case studies, sustainability was the starting point and goal of the sustainability concepts. In this study, one of the selection criteria to select the case studies was their sustainability goal. However, our analysis did not monitor the effective sustainability impact in the agri-food system. Therefore, future research could focus on the link between success and failure of collaboration of the experiments and the level of common commitment among the actors concerning their vision and principles of the concept of sustainability, as it is normative and contested. To do so, the narratives of the actors should be analysed. Moreover, it would be interesting to go one step further and compare the narratives of the actors of the sustainability experiments with the narratives of key actors at the agri-food regime level and link this comparison to the impact of the sustainability experiments at agri-food system level.

6. Conclusions

In this paper, we developed a new sustainability experiment systems approach (SESA) that consists of a multilevel and -dimensional analytic framework. The SESA framework combines criteria from a business as well as a learning perspective to evaluate multi-actor self-governing networks including entrepreneurs as supply-chain actors. We performed a cross-case analysis of four sustainability experiments in the Flemish agri-food system to test and validate SESA and identify key factors of collaboration success. The SESA framework was used successfully to analyse sustainability experiments with horizontal and vertical actors of the agri-food system.

The result of the cross-case analysis are 13 key factors of collaboration success, namely asset specificity, intensity of control, risk sharing, relevance of identify, length of relationships, trust, power, allocation of decision rights, frequency of interaction, information exchange, quality of communication, expected social gains, and motivation.

To conclude, the sustainability experiment systems approach (SESA) could be used as an assessment tool by researchers studying sustainability initiatives. Furthermore, results are promising for developing both a policy tool for policy makers to design, support or evaluate new sustainability experiments and a self-assessment tool for practitioners to design and reflect upon their sustainability experiments. Future research could focus on more mature sustainability initiatives to further evaluate the business criteria, on the influence of sustainability narratives of the actors on the collaboration and impact of the experiments and on the development of a self-reflection assessment tool.