Success in Transdisciplinary Sustainability Research

Abstract

The complexity of sustainable development and societal transitions require both analytical understandings of how coupled human-environment systems function and transdisciplinary science-to-practice approaches. The academic discourse has advanced in developing a framework for defining success in transdisciplinary research (TDR). Further empirical evidence is needed to validate the proposed concepts with TDR case studies. This paper applies a widely used TDR framework to test and critically evaluate its design principles and criteria of success with five TDR case studies the author is intimately familiar with. Overall, the design principles of the framework are validated for the five cases. Additional design principles are derived from the case analysis and proposed to complement the applied framework: (1) A project origin from society as opposed to with and for society; (2) Quickly available initiation funding; (3) Flexibility in time, objectives and methods throughout the research process; (4) Acceptance of process vs. project results; (5) Inclusion of public science communication; and (6) A demand-driven transition to a prolonged or new project partnership. The complementing principles are proposed for integration in the applied framework and are subject to further empirical testing. The reflexive empirical approach I have taken in this paper offers a key step towards removing institutional barriers for successful TDR, demonstrating how conceptual frameworks can be applied.

1. Introduction

1.1. Context and Aim

In view of facing a global human population of nine billion by 2050, both scientific methods and practically feasible solution strategies are urgently needed to enable resilience and sustainable co-development of society and nature [1,2,3,4,5]. Sustainability science needs to meet the challenges of assessing and understanding complex and dynamic social-ecological systems (SES) [1], while applying new tools of integrative transitions modeling [6] and transdisciplinary (TD) transformational research [7,8,9] with a clear place-based focus [10]. Emerging sustainability science can be subdivided into science for sustainability and science of sustainability [11]. The former is supposed to deliver answers to the pressing problems and societal challenges of today and tomorrow, such as climate change vulnerability, energy demand, and the provision of ecosystem services [8]. The latter is characterized as a rather monodisciplinary science that searches for a “generalizable scientific understanding of sustainability” [11].

TD is a ‘‘critical and self-reflexive research approach that relates societal with scientific problems; it produces new knowledge by integrating different scientific and extra-scientific insights; its aim is to contribute to both societal and scientific progress” [12]. TD research (TDR) in sustainability science involves both the exploration of new options for solving societal problems and the development of new methods and scientific innovation. It can be understood as a useful approach to tackling a societally relevant issue of complex nature [13]. TDR can be distinguished from action or applied research by its reflexive component [14].

TDR is problem-driven, starting from a real-world problem, integrating multiple stakeholders early on as experts who jointly define research questions, and producing practically relevant results that are widely communicated [15,16] in a reflexive setting on the TDR process itself [14]. TDR changes the culture or “rut” of producing knowledge—the goal is, as [17] (p. 532) states, “science with society” as opposed to “science for society”.

The growing literature on TD identifies a number of challenges in conceptualizing and executing TDR, specifically, determining how TDR projects are initiated and framed, how practitioners are involved, how quality standards are defined, how established academic disciplines are integrated with practical knowledge and what new methods are capable of achieving such integration, what type of outreach and communication is relevant and generates impact, how TDR is funded, and how progress and success are generally measured and defined [8,12,15,18,19,20].

The definition and assessment of quality and success in TDR is an overarching open issue and remains an open challenge [20]. The process from societal problems to the formation of a common research object and the extent of involvement of practitioners are prominent features of defining success in TDR, distinguishing it from other types of collaborative research. The extent to which integration and empowerment are achieved is viewed as one indication of success [12,14,17,19,21]. Lang et al. [8] discuss typical challenges and outline possible coping strategies providing a TDR framework with design principles for progress and success; moreover, they identify “fundamental differences among TDR approaches conducted in different cultural contexts” but find a general applicability of their principles, while demonstrating the need “to turn them into an evidence-based set of principles” (p. 40), based on empirical research experience. Other authors, such as [15], propose as well that case studies may serve as tools for defining progress and success in TDR.

The main aim of this paper is to analyze a set of five TDR case studies (

Table 1. Phases and practices in the applied transdisciplinary research (TDR) framework of [8].

Societal Practice	Transdisciplinary Research Process	Scientific Practice
Societal problems	Phase (A) Formation of a common research object: Problem framing and team building (Design principles I–IV)	Scientific problems
Actor-specific societal discourse	Phase (B) Production of new knowledge: Co-creation of solution-oriented transferable knowledge (Design principles V–VII)	Scientific discourse
Results useful for societal practice	Phase (C) Transdisciplinary integration: (Re-)Integration and application of created knowledge (Design principles VIII–XIII)	Results relevant for scientific practice

) using the existing framework of [8], to test the applicability of their design principles for providing further empirical evidence for how to define success in TDR.

1.2. Establishing a Common Transdisciplinary Research (TDR) Framework

From an analysis of a sample of the academic literature on TDR from over 40 years, starting with the OECD conference in 1970, Brandt et al. [20] identify common features to provide an enriched definition of TDR: “TD is a reflexive research approach that addresses societal problems by means of interdisciplinary collaboration as well as the collaboration between researchers and extra-scientific actors; its aim is to enable mutual learning processes between science and society; integration is the main cognitive challenge of the research process” (p. 4). This definition is the consensus of their literature sample but Brandt et al. [20] agree that it does not provide practical guidance for stakeholders involved in TDR. TDR is described as a “process for mutual learning” [17] (p. 379) through which science contributes to decision-making and benefits from the inclusion of insights from practice. The underlying process of knowledge distribution and management is of key interest; both for scientists who wish to connect with practice, for practitioners who benefit from science, and for (academic) teaching. Along with the broader ongoing discourse and the growing popularity of TDR, there is a vivid academic discussion on establishing a common TDR framework [8,12,15,17,19,20,21,22,23,24,25,26].

Existing frameworks provide specific principles for designing successful research processes. The systemic integration of reflexive processes in such frameworks seems to be an aspect of outstanding importance [18]. A TDR “outcome spaces framework” is proposed using a back-casting approach that begins at the end, or in the future, to draw a picture of where we want to be at a defined point in time [26]. The TDR framework proposed by [15] is described by five focal areas (inclusion, collaboration, integration, usability, reflexibility) and three phases (formulate, generate, evaluate), where reflexive processes and integration occupy an important role.

With specific relevance for this paper, the widely discussed TDR framework of [8,21] indicates three phases of the “ideal” TDR process, originally formulated by [27] and adopted with a more explicit practitioner involvement by [12] (Table 1). Phase (A) is the formation of a common research object; phase (B) is the production of new knowledge; and phase (C) is transdisciplinary integration. These three main steps are framed by societal and scientific problems affecting step A, societal and scientific discourse affecting step B, and results for societal and scientific practice affecting step C. The framework is based on the proposition that TDR feeds from two main approaches for linking societal problems to scientific ones: either society employs science to find solutions to practical problems (life-world approach) or science mainly follows its own goals in addressing societal problems (inner-scientific approach) [16,25,28]. The specific design principles I−XIII of the framework structured by the three phases A−C are shown in Table 3a–c.

The first phase of the TDR process (A), i.e., the transformation of a societal problem into a scientific one as part of the formation of a common research object, requires a reflexive process to ensure close ties between the academic and social stakeholders and processes for generating solutions to ensure a successful research outcome [8,12,27]. An “ideal” TDR process often does not function linearly, but phases A−C may have to be repeated to ensure integration between the life-world and the inner-scientific approach, a crucial task for successful TDR [8,12]. Being still a conceptual suggestion, Lang et al. [8] find that their framework requires further empirical testing for its establishment. In this paper, I, the author, describe and analyze five TDR case studies that I led for the identification of progress and success factors, applying and evaluating the framework of [8]. This framework is chosen due to its impact in the field; it is amongst the most widely applied TDR frameworks and the authors’ team involves some of the currently most cited scholars in this field.

2. Materials and Methods

The main goal of this paper is to apply and critically evaluate an existing TDR framework to analyze five TDR case studies. The methodological approach relies on previously published work on evaluations of case studies, such as [9,13,29,30], by formatively evaluating and testing the existing TDR framework of [8] with the empirical analysis of five TDR case studies (

Table 2. Overview on the five analyzed TDR projects.

Project Number	1	2	3	4	5
Project title	Wilderness conservation in Kamchatka, Russia	Ecotourism in Svalbard, Norway.	ClimAlpTour	ArcAlpNet	Brand building in the Swiss Gotthard region
Project duration (year)	2003–2006	2007–2010	2009–2012	2011–2013	2012–2013
Project topic	Assessing the potential for wilderness preservation in Kamchatka through the development of ecotourism	Analysis whether a diversification of the non-motorized tourism activities offered to date had the economic and political potential to increase ecofriendly tourism businesses.	Assessing the vulnerability of tourism regions in the Alps to climate change and development of adaptation strategies	Comparative resilience assessment from a network governance perspective in the Alps and the Arctic.	Measuring the emotional values of visitors and inhabitants in the San Gottardo region and developing a new survey tool
Financier	Kamchatka Ecotourism Society (KES), WWF Arctic Program, UNDP	Svalbard Villmarkssenter (SV), private funds.	Interreg Alpine Space program IVb	Swiss Network of International Studies (SNIS).	Program San Gottardo 2020 (PSG)
TDR team: #, type and timing of involved practitioners	One person from the WWF and one from the KES became part of the team with the joint project initiation	Two persons from the management of SV initiated the project and remained part of the team.	In the Swiss study region in the canton of GR, 3 practitioners (hotel, cafe, snow school) became part of the team during the first project workshop	In both case regions a minimum of two people from the local destination organisation and from the municipality became part of the team with confirmed funding.	Two persons from the management of PSG initiated the project and remained part of the team
Publications/further information	[30] Video: http://vimeo.com/ 5824277	[31] https://www.youtube.com/watch?v=_I00Q8W_wl4	[32] www.climalptour.eu	[33,34,35,36] http://www.pecs-science.org. http://arctic-alpine-resilience.net.	[36] http://www.gottardo.ch.

). I evaluate the cases alongside the three main phases, A, B and C, and their design principles in how far they empirically match with the framework of [8]. As proposed in the framework, I first describe and interpret each case alongside the three main phases and their nine design principles, and the three general design principles (Section 3). In a second step, I analyze each case in a way where the challenges throughout the research process, proposed solutions, and resulting criteria of success as published in [8] are matched and compared with the challenges experienced in the five TDR cases of this study. Matching challenges are confirmed, new or different challenges are described, followed by a discussion of the potential reasons for their occurrence (Section 4). After this analytical phase, I discuss if the design principles of [8] are applicable for designing successful TDR on the empirical example of these five cases. I finally conclude, based on the challenges and success factors identified in the cases, if and what additional design principles should complement the framework (Section 5 and Section 6).

For the analysis, I select five TDR case studies that I led over the course of ten years together with different research teams (Table 2). All cases describe TDR in the field of social-ecological systems, resilience and tourism in mountain communities of colder climates.

I deliberately select cases with which I am intimately familiar, bearing both advantages and disadvantages. I argue, however, that the advantage of in-depth understanding of the cases and the resulting comparability outweighs the possible disadvantage of potentially placing my own and the teams’ work into a better light (see also [9] for a similar approach). My goal is to test the framework of [8] for designing successful TDR, and the TDR cases I led are the empirical instrument to do so; there is no motivation in possibly setting them into a different light, since I have no stake in the framework of [8], and the outcome of this study for me is totally open. I am the person who can be most reflexive about these cases, which is an advantage for their evaluation.

I describe the obtained results from my (I) and the teams’ perspectives (we/us), while “team” clearly constitutes the inclusion of at least one of the practitioners and his/her perspective as part of each TDR project (Table 2). For each case description, at least one scientist and one practitioner team member have crosschecked the analysis and their interpretations. Forming TDR teams with scientist colleagues and practitioners allows for reflecting and sharing the synergies of methodological and abstraction skills of us scientists with the place-based knowledge and experience of practitioners. The reflexive exchange both for societal and scientific practice (Table 1) was designed throughout the project phases and clearly distinguishes these TDR cases from applied research [14].

3. Case Study Description and Interpretive Analysis

The cases I am presenting here share a common theme of tourism-dependent communities in mountain geographies and their resilience to environmental change. I describe and interpret each of the five cases alongside the three main phases of the framework and its design principles [8], and summarize the outcomes in

Table 3. (a) Descriptive overview of the five projects analyzed for phase A and its TDR design principles of [8]; (b) Descriptive overview of the five projects analyzed for phase B and its TDR design principles of [8]; (c) Descriptive overview of the five projects analyzed for phase C and its general TDR design principles of [8].

(a)
Project Number	1	2	3	4	5
Phase A The formation of a common research object
I. Build a research team	Scientists from relevant fields were included, as well as local practitioners and interest groups (Table 2). The TD research team was led by the author.
			Practitioners included only in phase B after researcher-driven project setup.
II. Joint problem framing	Defined by practitioners, picked up by the scientists	Framed by practitioners who involved scientists later	Framed by scientists without former involvement of practitioners	Framed by scientists based on results/partnerships from 3 and 4	Framed by practitioners who then involved scientists
III. Collaboratively defining the boundary/research object	yes	yes	Initially defined by the scientists and changed during the project due to late practitioner involvement	yes	yes
IV. Design a methodological framework for collaborative knowledge production	Close and reciprocal stakeholder involvement early on apart from project 3
	Multi-criteria analyis and expert interviews.	Multi-criteria analyis and expert interviews.	SWOT analysis, focus group workshops, social network analysis	Quantitative and qualitative social network analysis, focus group workshops, expert interviews.	Joint workshops with stakeholders and online survey
(b)
Project Number	1	2	3	4	5
Phase B The production of new knowledge
V. Appropriate roles	Practitioners were integrated, participating actively in all methodological steps.
			From after the first workshop on practitioners were partly the research subjects; their integration was analyzed.	Practitioners were research subjects, their integration within the local social networks was surveyed.
VI. Application of methods and settings for integration	Practitioners were involved in each methodological step and continuous integration was guaranteed by close collaboration and discussions with the research team leader.
			yes, but only after having adapted the initial objectives and having been flexible enough to employ new methods
VII. Capabilities for participation	Project initiation as response of the leading scientist to an expressed societal problem.		Research subject was participation (SNA)		Same as projects 1 and 2
(c)
Project Number	1	2	3	4	5
Phase C Transdisciplinary integration
VIII. Two-dimensional integration	Implementation due to local cultural limitations not initially possible.	Due to a lack of local cultural support no implementation on a larger scale was possible.	Implementation prepared through policy recommendations and integration of policy makers. Scientific integration achieved through publications.		Implementation directly executed in a new regional brand. Scientific integration achieved through publications.
IX. Targeted ‘products’	Personal discussions reported the project progress to the local stakeholders, a short movie was produced to communicate and advertise the project outcomes, and a conference presentation has been done.		Local final workshops reported the results, and a final project policy report in the local language has been published.	A website, newspaper articles, a short movie, and a policy report have been published and the results discussed with local policy makers.	A project report has been published and the results discussed with the practitioners.
X. Scientific and societal impact	Goals to generate results were achieved and tested with market demand but cultural reasons did not permit further scaling of implementation. Prolonged partnerships with further TD research would support this process.		Achieved (see detailed descriptions in Section 3).
General principles
XI. Facilitate continuous evaluation	Some of the proposed practices have been successfully implemented five years later.	Close relationships with stakeholders and the follow-up project ArcAlpNet.	Adaptation strategies are being implemented, a new follow-up project has started.	The funding agency demands annual reports which control the evaluation. Practitioners are subject of this research thus subject of evaluation.	A follow-up project from another team has implemented our findings and developed a new regional brand which is now in use.
XII. Mitigate conflict constellations	No substantial conflicts within the scientists group or the whole research team, including the links with practitioners, were encountered due to the experience and flexibility of the whole team.
XIII. Enhance capabilities and interest for participation	Initiation from practice ensured interest in participation; capabilities needed to be addressed and supported by the TD skillset of the scientists. Flexibility, broad interests and communication skills were important.
			Initiation without practitioner involvement led to lack of participation. Later adoption of real demand led to more participation, and a follow-up project; topical, methodological, timely and financial flexibility were important.

a–c. I look at both the practical (life-world) and the knowledge (inner-scientific) aspects for applying and evaluating the framework, while I place special focus on my own reflexive role as the project leader in the five cases. Following the framework, I describe and interpret societal and scientific problems of phase A, societal and scientific discourse of phase B, and results useful for societal practice and relevant for scientific practice of phase C.

3.1. Case 1—Wilderness Preservation after the Political Decay in Kamchatka, Russia (2003–2006)

3.1.1. Phase A. The Formation of a Common Research Object

This project partnership was jointly initiated in 2003 after I watched a TV report by the World Wildlife Fund (WWF) and United Nations Development Programme (UNDP) on their efforts to incubate and support wilderness preservation on the Siberian peninsula of Kamchatka, Russia. The main problem was described as fast and uncontrolled economic development after the political decay of the Soviet Union, such as the fishing, forest, and mining sectors threatening the amazingly rich ecosystems that were preserved during the complete closure of the peninsula as a strategic nuclear base during the Cold War. The TV report mentioned the problems of finding suitable alternatives that would allow for sufficient economic development without threatening wilderness areas, including an obvious call for external support. The WWF reported a partnership with UNDP and the Kamchatka Ecotourism Society (KES) to establish one of the first protected areas, Nalychevo, whereas the acceptance of locals for behavioral restrictions and the financing of the park were described as the main barriers for its successful establishment. One potential solution for financing the park was the development of ecotourism, which was initiated by the construction of a lodge near a thermal water area. An open issue that persisted was determining how a functioning ecotourism industry could be established and what the economic potential thereof would be. During a talk with the responsible project manager of the WWF, we jointly outlined the objective of ecotourism development: analyzing whether attracting tourists willing to pay more for ecofriendly and socially responsible trips both in the summer and winter, especially in the latter, could help to drive the local system in a more sustainable direction. The local conditions for terrain, infrastructure, logistics, and public support, as well as the overall (global) market potential were unclear and subject to research.

The formation of the research team was straightforward. After my initial email contact with the WWF, I was introduced to the manager of KES, and we quickly came to a joint definition of the problem and the TD research goals. The establishment of this partnership was supported by my active search for and offer of joint research, knowledge and experience in the addressed topics, and some available funding to initiate this project, i.e., establish contacts and develop a funding proposal.

3.1.2. Phase B. The Production of New Knowledge

We developed a joint research proposal involving the WWF, UNDP and KES. Funding was provided by these project partners for both a market potential analysis and a local study visit, including a SWOT (strengths, weaknesses, opportunities and threats) analysis of the local potential for ecotourism in winter, participative workshops with local stakeholders from among park officials, the KES, WWF, the local university in Petropavlovsk, outfitters and guides, politicians, and for the planning and organization of a pilot trip. In the winter of 2005, we developed and tested a preliminary ecotourism product with an international group of skiers from Europe and North America, evaluated the results, and discussed the outcomes and further steps with the local stakeholders, as well as with travel agencies from abroad. The regional natural potential for such ecotourism offered year-round, as well as the international market potential worldwide, proved to be substantial [31].

Challenges in the societal discourse were the differences in cultural understandings, e.g. of how we as scientists and field testers (tourists) were used to being treated. We were never left without a local “guide” who observed and controlled what we did and where we went. The scientific discourse was steered by the needed flexibility in our time schedule, as well as the methods and their application, since we needed to adapt to spontaneous demand and events of the local partners.

3.1.3. Phase C. Transdisciplinary Integration

● (a) The societal practice

Already during phase B, when tourism products were jointly designed and tested, societal practice received benefits from the TD partnership. At the end of the project we discussed the results in collaborative workshops together with policy makers, and were featured in local newspapers. We produced a movie clip for a public audience, especially for travel agencies, outfitters, and tourists to inform them about sustainable tourism development and offers in Kamchatka, which went viral on YouTube [37]. At another conference about ten years later, I saw in a poster presentation by the University of Petropavlovsk, Kamchatka, that some of the TDR project outcomes had meanwhile been implemented.

● (b) The scientific practice

I presented the results at a conference [31] and used both the results and the experience of forming a TD research team from this project for the setup of project 2.

3.2. Case 2—Economic Development and Wilderness Preservation on Svalbard, the Arctic of Norway (2007–2010)

3.2.1. Phase A. The Formation of a Common Research Object

I initiated the project at the 2007 Global Ecotourism Conference (GEC) in Oslo after having presented the results of the previous Kamchatka project, case one. At the conference, politically engaged owners of a dog sledding company in the Arctic of Svalbard discussed a problem similar to that reported by the WWF in the TV report from Kamchatka. A member of the family-owned dog sledding business, Svalbard Villmarkssenter (SV), asked me whether a similar collaborative research partnership could be initiated to tackle a similar problem and research question. SV had been politically active in the preservation of the Arctic wilderness on Svalbard, protecting areas from motorized traffic, mainly from snow-scooter tourists, which has become a serious problem due to the growth in tourism numbers—by a factor of ten over the past ten years—and has begun to threaten wildlife habitats and the fragile Arctic ecosystem. The local politicians hesitated to protect more non-motorized areas due to the economic importance of snow-scooter-based tourism, and SV was facing a lack of further support in setting up new regulations.

The jointly developed objective of a new research partnership with SV was to analyze whether a diversification of the non-motorized tourism activities offered to date had the economic and political potential to increase ecofriendly tourism businesses. A best-practice model of how economic development could be decoupled from local natural resources depletion was to be developed, attracting more eco tourists with a higher willingness to pay for such responsible activities, and thus potentially substituting conventional tourism services. Such a model could then create push and pull factors to drive the tourism-dependent local economy to become more environmentally friendly, demonstrating to policy makers that protecting more land area from motorized traffic would not threaten economic development, and to other operators that ecofriendly activities can be a viable business alternative. Research goals involved a world market analysis for such types of offers to the Arctic, as well as a local analysis of the natural, social, and economic potentials of such diversified ecotourism products.

The experiences of the previous project, case one, helped to initiate this new TDR partnership; especially the needed flexibility in adapting one’s own methods and timeframe was again of key importance.

3.2.2. Phase B. The Production of New Knowledge

In the following year, we visited Svalbard for an initial participatory workshop with SV and other stakeholders to collect data for the analyses and to set up a pilot test for a new ecofriendly trip based on dog sledding, backcountry skiing, wildlife observation, and snow kiting. We developed a training program for SV to be able to accommodate the needs of tourists booking such activities. The trip was successfully completed, and the analysis delivered high local and general market potential for such activities [32]. During this TDR work we enjoyed substantial time together with the SV family on joint day trips and informal discussions. Their and our—the scientists’—socially open characters and shared love for the Arctic outdoors helped to form a team beyond professional boundaries and to become sort of friends. This close relation allowed for more openness in discussing and sharing both local knowledge and science knowhow, and for an ongoing, rich discourse as the core of this TDR project.

3.2.3. Phase C. Transdisciplinary Integration

● (a) The societal practice

We discussed the project results with the partners in collaborative workshops together with policy makers at both study sites. We published a policy report and were featured in local newspapers. We (the team together with local tourism stakeholders) designed new sustainable tourism packages and produced a movie to help advertise the packages via travel agencies in central Europe and North America in the following years; the movie went viral on YouTube [38].

● (b) The scientific practice

I presented the results at a conference [32]. The conceptual insights in the topic of community resilience, the research experience and new contacts helped to develop project 4, specifically the Svalbard case.

3.3. Case 3—Adaptation to Climate Change in the Alps, Swiss Surselva Region (2009–2012)

3.3.1. Phase A. The Formation of a Common Research Object

The European Interreg Alpine Space project ClimAlpTour (www.climalptour.eu) was a pan-European project from 2009 to 2012, set up by 17 European partners from regional administrations, NGOs, research institutes, and universities [33]. The goal was to develop adaptation strategies for tourism in Alpine countries to adapt to climate change, based on an impact analysis of a collection of climate and economic data for 14 test sites throughout six Alpine countries. One of the main work packages involved the development of adaptation strategies in participative stakeholder workshops, such as product diversification for snow-free tourism activities. Other work packages involved a Delphi analysis, awareness-raising, and public outreach. Local stakeholders in the test sites were less involved in the initial development of a common research object in terms of becoming part of the TDR team from the beginning on. This project was mostly set up by scientists in a comparative alpine scope, while local practitioners were included from the first workshops on, some of them becoming then a close part of the TDR team (Table 2). I came to the project after its start and joined the first workshop without having planned for it.

The integration process and the reflexive exchange qualifying this project as TD started later than in the other four cases described in this section. The reflection with practitioners after the first workshop led to a change in the direction and methods in this project. The feedback from the workshop participants made me steer the remainder of the project in another direction as set up by my colleagues. Only from after this first workshop on, did the involved local practitioners actively steer the project in the direction with the greatest usage for practice. I was not proficient in SNA methods when I proposed to execute the social network analysis (SNA). Together with my university colleague, we acquired the needed skills within the scope of this TDR project, which required a lot of time and overall flexibility.

3.3.2. Phase B. The Production of New Knowledge

The project was successfully completed, yielding the expected results of concrete adaptation strategies for the 14 test sites, including public and policy-making workshops at the end. In our Swiss test site, though, the original objectives had to be changed after the initial stakeholder workshop and the first inclusion of practitioners in the TDR team. Practitioners expressed different local problems than initially understood by the scientists, especially the problem of little collaboration in the region. They asked us to address this problem that they expressed to be more relevant than climate change. Only then did I decide to execute an SNA to set the basis for the current state of collaboration. The self-reflexive character of this method with the visualized social networks were the core of the joint production of new knowledge.

3.3.3. Phase C. Transdisciplinary Integration

● (a) The societal practice

We discussed the project process and results with the partners in collaborative workshops together with policy makers and were featured in local newspapers. We published a policy report [32] and made workshop contributions. The larger ClimAlpTour project team designed a website featuring a tool for developing adaptation practices designed for practitioners (www.climalptour.eu). In our study region of eastern Switzerland, some of the identified adaptation measures were accepted for implementation by local policy makers and private businesses.

● (b) The scientific practice

We published the results in the academic literature [35,36] and made conference presentations. The integration and collaboration with practitioners was a self-reflexive process due to using the SNA method to map stakeholder integration. The methodological insights in social network analysis and new contacts helped to develop project 4, specifically the Surselva part.

3.4. Case 4—Comparative Network Governance and Community Resilience in the Arctic and the Alps-ArcAlpNet (2011–2013)

3.4.1. Phase A. The Formation of a Common Research Object

The experiences of the two previously described projects, cases two and three, were funneled into this new project: both cases were comparable given the vulnerability of the communities to climate and further environmental change in both regions, their dependencies on the tourism-based economy, and the identified lack of social collaboration for coping with complex problems. However, both places in Surselva and Svalbard were completely different given their cultural backgrounds, fluctuation of (temporary) inhabitants, and internationalization rates. We thus developed a new research plan for a comparative study of social network governance and community resilience to climate change, deeply rooted in the local stakeholder networks, and involving quantitative and qualitative social network analysis. The existing collaborations with practitioners at both sites were successfully transformed into this new TDR team.

3.4.2. Phase B. The Production of New Knowledge

ArcAlpNet (http://arctic-alpine-resilience.net) is ongoing at the time this article was finished. ArcAlpNet is financed by the Swiss Network of International Studies (SNIS) and involves research partners from Switzerland and Norway, and stakeholders from the field in both study regions (Table 2). Based on the contacts made in the two former projects, and based on the first SNA in Surselva, we performed another SNA on Svalbard, supported by qualitative interviews with selected actors from the local communities in both regions. Results have been published in the academic literature [33,34,35,36]. In summary, we observed signs of both lower and higher resilience in both areas, whereas an overall lack of collaborative ties exists in both regions, but more so in Svalbard. Innovative capacity is higher on Svalbard than in Surselva. Both aspects relate to the contexts of the places, where higher internationalization on Svalbard leads to more innovative ideas; higher fluctuation leads to lower feeling of attachment or belonging to the place and thus a lower sense of collaboration. In Surselva, low fluctuation and low influx of fresh ideas are related to a type of lock-in effect [35,36].

In-depth knowledge of both cases and established contacts with local practitioners who were satisfied with the initial TDR partnerships were success factors for this project.

3.4.3. Phase C. Transdisciplinary Integration

● (a) The societal practice

We discussed the project results with the partners in collaborative workshops together with policy makers in both study sites. The project and the summarized outcomes were featured in local newspapers. On Svalbard, the local destination management organization confirmed that they would support the development of a common vision for increasing formal collaboration. In the Surselva-Gotthard region, some more peripheral stakeholders (with fewer formal collaborations than the more centralized core stakeholders) began to form a new sub-network with the goal to increase formal collaborations in the region.

● (b) The scientific practice

We published pure science articles in the academic literature [33,34,35,36]. We made conference presentations and workshop contributions, designed a website (http://arctic-alpine-resilience.net) and a Facebook page; together with photographs from both study sites, these pages support the viral spread of information about the areas. We produced a short movie clip made available on YouTube. Furthermore, the project has been linked to the Program on Ecosystem Change and Society (PECS, http://www.pecs-science.org), a ten-year initiative jointly sponsored by International Council for Science (ICSU) and United Nations Educational, Scientific and Cultural Organization (UNESCO), and contributing to the Arctic Resilience Report (ARR).

3.5. Case 5—Regional Economic Development and Emotional Brand Building in the Gotthard Region (2012–2013)

3.5.1. Phase A. The Formation of a Common Research Object

This research partnership was initiated by the regional organization Program San Gottardo, enabling economic cooperation and development within the Swiss Gotthard perimeter, connecting four cantons [39]. San Gottardo learned about the team’s work through the SNA in the ClimAlpTour and ArcAlpNet projects as one identified actor and asked to support the San Gottardo brand-building process in the region. Since this practitioner stimulated the TDR project and since we, the scientists, were flexible enough to employ methods suitable to address the question for emotional brand building for San Gottardo, the formation of this new TDR team was no problem.

3.5.2. Phase B. The Production of New Knowledge

Supporting the strategic plans of San Gottardo, we developed a survey tool to measure the emotional values of visitors and inhabitants from the four cantons towards the San Gottardo region and a new regional brand “San Gottardo”. We supported the organization San Gottardo in local stakeholder workshops in the four cantons to match the “external view” of visitors and inhabitants with the “internal view” of local businesses and professionals on a new regional brand. During these workshops, we collected additional data on the business actor networks to analyze collaboration within and beyond the cantonal borders of the San Gottardo perimeter, to support the brand-development process with improved collaboration and to feed back into the ArcAlpNet project, in which further social network data of the Surselva region would help to fulfill the objectives of the project. As the scientists, at the time of the project start, we were not proficient in the measurement of emotional brand building and picked up an initiative suggested by the practitioner.

3.5.3. Phase C. Transdisciplinary Integration

● (a) The societal practice

Throughout the TDR process, we, the scientists and the directly involved practitioners from the regional development agency PSG2000, exchanged our viewpoints on the methodological approach in the project, and we (the scientists) explained why the selected methods were used.

The overall project results were discussed in a stakeholder workshop together with local policy makers. Since then, other teams have used the results to develop the new regional brand San Gotthard, which is now in place [40].

● (b) The scientific practice

We published the results in the academic literature [37,38,40,41]. Throughout the project, the adoption of a new methodology for measuring emotional brand value stimulated by practitioners had been an iterative learning process.

4. Analysis of Challenges and Success Criteria

I now analyze and evaluate if the challenges and proposed criteria of success in TDR expressed in [8] (Table 3, p. 33) are valid for the five cases. I first evaluate which challenges we identified in each case, following the structure of the framework. For each identified challenge I propose a solution, and from these solutions I derive factors of success. I then discuss if these identified success criteria match those in the framework; the ones that are not covered in the framework are proposed as complementary design principles for measuring and designing success of TDR (Section 5). The identified challenges and proposed success factors of the five cases are summarized in

Table 4. (a) Identified challenges, solutions and criteria of success in the five analyzed projects evaluated by phase A of the design principles of [8]; (b) Identified challenges, solutions and criteria of success in the five analyzed projects evaluated by phases B, C and the general aspects of the design principles of [8].

(a)
Proposed Phases and Principles [8]	Identified Challenges	Proposed Solutions	Criteria of Success
Phase A The formation of a common research object
I. Build a research team	Timing of the involvement for developing a research team is important. Wide TD skillset of the researchers is needed.	TD skillset of the research leader and of the team. Quick initiation funding to invest more time at the beginning before knowing exactly what the project would look like.	TD skillset of the research team. Quickly available initiation funding.
II. Joint problem framing	Scientist-driven research ideas need to be ‘sold’ and adapted to the need of the practitioners. Risk that project is not funded may disappoint partners. But if not included early enough, the project may risk not addressing the real need. Joint problem framing process is time consuming and needs to be financed with low risk and high flexibility.	Real problems incubate new research partnerships; ideally, a project naturally occurs as a follow-up from a previous project, or stakeholders express interest to develop a project together. Scientists need to understand problems and language of practitioners. Integration must be flexible. Investing time before the project start is important.	Project initiation from and with society. TD skillset of the research team. Flexibility in time, content and methods. Quickly available initiation funding.
III. Collaboratively defining the boundary/research object	Failure due to cultural and other barriers as ‘normal’ and acceptance of some sort of failure and another time scale for identifying success since implementation of project results may take time.	Not relevant if the project is initiated from society by real demand.	Project initiation from and with society. TD skillset of the research team. Flexibility in time, content and methods. Quickly available initiation funding.
IV. Design a methodological framework for collaborative knowledge production	Time consuming and socially (skills, matching with needs of practitioners) challenging.	TD skillset of the team and time to engage properly in the development of the framework are important.	TD skillset of the research team. Quickly available initiation funding.
(b)
Proposed phases and principles [8]	Identified challenges	Proposed solutions	Criteria of success
Phase B The production of new knowledge
V. Appropriate roles
VI. Application of methods and settings for integration	The research team needs to be able to apply a wide range of TD methods.	TD skillset of the team is important, and time and flexibility to tune the integration of methods and to improve collaboration.	TD skillset of the research team. Scientific rigor. Flexibility in time, content and methods. Main funding smoothly available and based on success of initiation funding.
VII. Capabilities for participation	Time consuming and socially (skills, matching with needs of practitioners) challenging.		TD skillset of the research team. Disciplined interdiscplinarity. Scientific rigor.
Phase C Transdisciplinary integration
VIII. Two-dimensional integration	Not identified as a challenge in these case studies.		Project initiation from and with society. TD skillset of the research team.
IX. Targeted ‘products’			Satisfaction of the practitioners with the project. Public outreach—science communication reaching a larger public audience. Main funding that includes a budget for public media production and outreach. Accepting process versus product results.
X. Scientific and societal impact	Process and result outcomes need to be communicated in a more appropriate way to a larger audience than achieved with pure science articles.	Viral outreach with other types of media, e.g., movies, photography, art, can help to reach a larger audience and to have a greater societal impact.
General principles
XI. Facilitate continuous evaluation	Not identified as a challenge in these case studies	Not applicable in these case studies.	Satisfaction of the practitioners with the project. Public outreach—science communication reaching a larger public audience. Transition to a follow-up project partnership requiring further quickly available transition funding.
XII. Mitigate conflict constellations			Project initiation from and with society. TD skillset of the research team. Flexibility in time, content and methods. Quickly available initiation funding.
XIII. Enhance capabilities and interest for participation

a,b.

4.1. Case 1—Wilderness Preservation after the Political Decay in Kamchatka, Russia

The lack of infrastructure for transportation, accommodation, and logistics, and the difficult and long journey required for reaching Kamchatka proved to be major limitations for establishing a new ecotourism market capable of financing wilderness preservation in the new protected area. Cultural barriers and a lack of local acceptance of behavioral change were experienced, aligned with the corrupt political system shortly after the political decay of the Soviet Union. The secret service FSB, which replaced the former KGB, closely monitored foreigners’ footsteps and limited the group and the guide, e.g. when exploring volcanoes on skis; the group was never left alone, neither in the mountains nor in town, neither in bars nor in spas. Though partly exciting from a tourist perspective, this culture of control limited the possibilities for exploration as well as the group dynamics. The encountered limitations led to a time delay in further establishing ecotourism for the winter season. Seven years later, further progress has been observed, and new local initiatives have been spreading from the initial seed research project. Logistical, infrastructural, and technical limitations were among the problems expected. The main limitations to solving the real-world problem of ecofriendly financing preservation in Kamchatka with protected areas while enabling economic development were cultural and social barriers. Without the initial inclusion of local stakeholders, the project would not have been possible; thus, setting up a project that seeks to address regionally based real-world problems requires science-to-practice partnerships from the earliest stage. However, even the early and constant inclusion of locals is no guarantee for success—sometimes, it may simply be a matter of time before pioneering aspects are incubated and picked up to overcome a barrier, e.g., cultural values or pressures, thus preventing the further limitation of the process (Table 3a–c).

4.2. Case 2—Economic Development and Wilderness Preservation on Svalbard, the Arctic of Norway

Despite the encouraging results of this TDR partnership, the initiation of a new ecotourism economy on Svalbard did not lead to visible growth. Tourism packages were marketed and new trips were offered, but neither SV nor other local operators or other tourism businesses picked up on this development or pushed further projects and processes. We encountered a lack of real engagement and a lack of further support. We learned from personal discussions with local stakeholders that the policy recommendations were conflicting with the local culture of mobility. The advertisement of snow-scooter-free activities and the request for the protection of further areas from motorized traffic near the main settlement of Longyearbyen collided with the deeply rooted need for freedom and mobility. This need is strongly based on individual snowmobile use and access because there is only one short road in the settlement, and all other travel in the snowy environment occurs on snowmobiles three-quarters of the year. In addition, the fast-growing market for snowmobile tourists and the high competition of outfitters prevented joint collaborative action for a more “qualitative”, more ecofriendly economic development. The developed ecotourism packages are still in place and advertised, and an annual joint trip with clients has enriched the SV business portfolio. Only very recently have there been the first signs of a change in direction in the local tourism practices. It appears, however, that there are other factors associated with social interaction that require further research. Local experts from practice, which allowed for the engagement of scientists to help address the outlined questions, initiated this research project. Despite the need for such support and despite the objective fitting of the results of the research, certain cultural barriers or time limitations prevented the outcomes from reaching their expected potential. Research initiated from practice does not guarantee success in implementation because other complex systemic impacts—e.g., cultural preferences, or simply time and “the right moment”—may hinder the implementation of results. Instead, or in addition, to engage in the TDR process may be seen as a success in itself (Table 3a–c).

4.3. Case 3—Adaptation to Climate Change in the Alps, Swiss Surselva Region

In all test sites, participative workshops were held and functioned as planned—with the exception of one site, the Swiss Surselva-Gotthard region. In the first local stakeholder workshop in this region, we asked participants what their main problems were in the context of climate change. They responded that climate change and the development of adaptation strategies were not their main problem—though outlined by us and planned by the ClimAlpTour project for all test sites. The participants mentioned that the lack of collaboration and communication in the region was the main problem and a barrier for them. The participants of the initial Surselva workshop complained that their ideas and demands were not heard in the region and that local politicians and decision makers neglected the problems of the smaller, economically less important stakeholders. Interestingly, most of those economically more important actors did not attend the first workshop [36]. Thus, we decided to switch the planned topic of the workshops to help the stakeholders who demanded support, and we undertook a social network analysis (SNA) to define the status quo of collaboration as a basis for developing recommendations and action plans for improvements. We completed the SNA and discussed the results in the next workshop. Based on the results, we then held the initially planned workshops to develop adaptation strategies for climate change, employing the workshops as a preliminary tool for improving collaboration. Research on sustainability should be developed together with stakeholders; more importantly, the initiative should come from the stakeholders because in this project the planning researchers set up a process that addressed the main problem of climate adaptation in most study sites, but not in Surselva. If we had continued with the initial workshop plan, we would have missed the real problems of the region and likely would not have succeeded (Table 3a–c).

4.4. Case 4—Comparative Network Governance and Community Resilience in the Arctic and the Alps-ArcAlpNet

To date, we have encountered no limitations in the research process. ArcAlpNet provided in-depth insights into the social networks of the communities in the two regions, and provided us with new professional and personal contacts. In both regions, new projects or project proposal developments arose: in Surselva, the regional organization San Gottardo, enabling economic development and cooperation, initiated a follow-up project with the author by their own request, leading to case five described below; in the Arctic, the author has led a follow-up project with an extended SNA in indigenous communities. Collaborative research addressing real-life problems and involving local stakeholders will naturally lead to new partnerships and projects that should then address relevant problems and questions facing societies. If the initial research is successful, new research projects may automatically evolve. Thus, a measure of success of TDR can be whether a demand- or problem-driven flow of evolving projects from an initial engagement occurs (Table 3a–c).

4.5. Case 5—Regional Economic Development and Emotional Brand Building in the Gotthard Region

This project was successfully completed, and a follow-up research project with the same team was initiated by San Gottardo. We encountered no substantial limitations in the research process.

Most of the challenges included in [8] can be validated for this and the other four cases, although some of those challenges, especially in phases A and C, were not experienced in our cases. A major reason for this may be that the research problems in the five cases were a result of demand from practitioners (this has been the main reason for follow-up projects), as well as due to the fitting TD skillset of the research teams. TDR initiated by experts from the field may naturally lead to a follow-up project if a real problem exists, if the project is successfully completed and/or if the practitioners see the benefit in the pathway of the TDR project partnership. In a series of TDR projects, overlaps may occur and develop, in which data or results may be beneficial for other ongoing research, and thus create synergies that may incubate new TDR projects. Flexibility in the research direction adopted is needed (including the research question, methods, and research process); however, to be able to encounter and design opportunities, broad interests and experiences of skilled TD researchers must be harnessed to be able to identify opportunities.

5. Proposition of Six Complementing Design Principles

Based on the evaluative results summarized in Table 4a,b, I now present six criteria for defining success as of relevance for the five cases in this section. I evaluate these criteria with the design principles of the existing framework and propose them as complementary design principles. From the previous sections, I identify in total seven criteria of success with high relevance in the analyzed cases, but since the principle “TDR skillset of the research team” already exists in the applied TDR framework, it is not listed in the following complementing principles list again (

Table 5. Six complementing design principles to become embedded in the framework of [8].

Phases of [8]	Complementing Design Principles
Phase A The formation of a common research object
	Principle 1. Project initiation from society
	Principle 2. Quickly available initiation funding
	Principle 3. Flexibility in time, content, and methods
Phase B The production of new knowledge
	Principle 4. Acceptance of process vs. project results
Phase C Transdisciplinary integration
	Principle 5. Public outreach—science communication for a larger public
	Principle 6. Transition to a follow-up project partnership

). To highlight the importance of the skillset for success in TDR found in these cases, I summarize my findings in the next paragraph, Section 5.1, before listing the six complementary design principles starting with Section 5.2.

5.1. TDR Skillset

Initiation and progress in the described TDR projects were highly dependent on the skillset of scientists and practitioners of the team who had to be broadly interested in a wide range of topics and methods. Sometimes the main research focus needed to be expanded, and it was possible that such chances to engage in new research were not very obvious upon first glance for a more focused or disciplinary-thinking scientist. An entrepreneurial character, being able and willing to navigate between the academic and the professional world with their different languages, objectives, and goals, enabled the construction and bridging of TDR projects, which was also observed by [42]. The broader and more systemic the training, interests, and experiences of a TD scientist, the better equipped he or she will be to successfully adopt a complex project, while requiring support from “interdisciplinary thinking disciplinary scientists” [17] (p. 529) for certain questions.

Because sustainability science looks at the co-development of human and environmental systems, it feeds from methods and knowledge of the natural and social sciences. TD scientists thus ideally need to be trained in both the natural and social sciences, which requires more than being able to connect the disciplines; it requires being embedded in fields from both the natural and social sciences through early and profound academic education to be able to “communicate and investigate across disciplines” [17] (p. 535). Because sustainability science and corresponding TDR involve society and practitioners in the science process, a sustainability scientist ideally should also be able to connect to experts from practice and have individual professional work experience to be able to better understand the language and problems of the professional world and to incorporate the entrepreneurial thinking necessary for innovative, creative, and economic problem-solving. TDR requires new partnerships between science and practice and real life; ideally, scientists would thus be embedded in other tasks and have professional experience from outside of academia. Therefore, a new skillset is required, building upon academic training both in the natural and the social sciences, having experienced the professional world in the private and the public sector, being equipped with an entrepreneurial way of thinking, and topped by a set of communication soft skills. The importance of identifying and building a team with relevant disciplinary/interdisciplinary skills and “real-world” experiences has been identified by [21,42,43,44], who suggest that the training of TD scientists should focus on skills for collective learning processes. Pohl et al. [42] describe the different roles a TD scientist has to adopt—the roles of a reflective scientist, intermediary, and facilitator of a joint learning process—which need to be addressed in the training of TD scientists.

This demand for diversity in the skillset of a TD scientist, and/or of a research team, may conflict with the current higher education teaching system, which is designed to foster disciplinary, theoretical, and focused academic thinking and publication [39]; furthermore, it conflicts with the current science funding system, which is based on individual theoretical experience proven by peer-reviewed publications, neglecting other important skills. Lang et al. [8] (p. 40) confirm the need for “continuous structural changes in the academic system to build capacity for TDR...”, as do [44] (p. 240), who identify the need for new teaching formats to address the growing demand for TD researchers. Since this principle is already part of the framework [8], it is not suggested as one of the complementary principles.

5.2. Phase A. The Formation of a Common Research Object

(1) Project initiation from society

This principle complements the framework’s principles “Joint problem framing” and “Defining the research object” since it stresses an earlier involvement of practitioners: a joint definition as a societal problem is not needed if the project is initiated by and originates from society, because it will be by definition a societal problem. A joint definition as a research problem, is, however, needed and should remain an integral part of phase A.

In each of the five described TDR cases, new research partnerships and projects evolved driven by problems and practical demand. One project led to another, and a flow of research developed. I understand the five cases as a process, since the last four projects evolved from the first one. This insight may only be derived by myself since I have the reflexive position of seeing the cases this way. New projects were triggered from practitioners with concrete problems to solve; if real problems existed, new TDR was designed based on solving the specific problem. A key task for the involved scientists was to recognize such opportunities, to be able to communicate with practitioners in their language and to be open and flexible enough to support the practitioner in developing a valid research question starting from their practical problem. In a reflexive context, the practitioners in the TDR team needed to be open enough to understand and respect one motivation of the scientists as well, namely to benefit from a project in an academic sense of publishing (scientific practice). Setting up a TDR project has been a process of “selling” a research question to practitioners and funding bodies and supporting practitioners to understand the potential in collaborating with scientists. A challenge has been to convince the partners that investing in a partnership may only solve a complex problem in the medium to long term, a process that requires significant time.

A critical issue encountered in the early integration of stakeholders is the “equality” of practitioners as experts from the field with scientists in an integrated research process, which often is only theoretically encountered and makes TDR more demanding and time-consuming [28]. Maintaining flexibility in this type of collaboration and integration, which may change during the project, can help to address this challenge. However, the dynamic scope and timing of stakeholder integration may lead to difficulties in the definition and classification of TDR, which thus may have to allow for such flexibility in the scope and timing of collaboration and integration.

How TDR projects are initiated and completed have shown to be critical for their success. We observed better progress and fewer obstacles to overcome when an active demand from the field triggered the research partnership; the project setup was easier and faster, the willingness of the practitioners to support the research was higher, and the research process was smoother and faster. Still, initiation from practice did not guarantee success: sometimes cultural barriers or simply time delays led to obstacles that prevented a practical objective from being completely fulfilled or implemented or led to barriers in the research process, which was observed by [21] as well. The mental models of what a solution to a complex and often value-driven sustainability problem should look like are different, and so are the levels of satisfaction with the results. Whereas the scientist—often coming from outside the direct field of practice and context—implements a methodological process to perform an objective analysis of the problem while still being subjectively affected through his or her own mental model, the directly involved local professional may have different ideas, values, constraints, modes of thinking, and approaches to the same question. The communication process between science and practice is thus of utmost importance, and most important is the initiation phase of a project.

Time is needed to build trust and understand each other’s mental models before the project is initiated to ensure a fit in the overall project goals and setup. Lang et al. [8] (p. 29) come to a similar conclusion, highlighting the collaborative definition of boundary and objective of the research, though asking for the “joint definition of the sustainability problem as a societally relevant problem”. This indicates project initiation by the scientist, whereas the case studies presented here highlight the success of research initiated by society, in which the scientist helps to frame the problem—but a joint definition as a societal problem is not needed if the project is initiated by society because it will be by definition a societal problem.

TDR should thus not only be research “with” society as [17] (p. 532) formulates it, but even research “from” society. The notion of research from society clearly indicates that ideally the problem expression and the demand for support in TDR should originate from practice. Most current funding schemes, instead, demand TD scientists to come up with a developed detailed plan including partners from practice to do the research. Then scientists are asked to define the problem as socially relevant. A key task for TD scientists will therefore be to identify, contact, and approach (potential) initiators (practitioners with a problem) of TDR projects and guide them through a joint research setup process. TD researchers need to remain flexible in the research direction to be able to accommodate obstacles and opportunities, which requires adequate funding tools. Scientists may still initiate TDR; however, the process of joint problem framing will then become even more important and critical for the TDR to be successful.

(2) Quickly available initiation funding

The development of a common research object and of trust within the TDR team requires sufficient time in phase A, ideally prior to the main project setup and the submission of a main research proposal. If funding institutions were accepting short pre-proposals with a quick review time, then TDR projects could invest more time in the critical setup phase of the research project. The more time and thus funds were available for an intense pre-project phase, the higher the chance for a successful main project phase. Thus, I propose a quickly available initiation funding for phase A but also from phase C to a new project as a complementary design principle.

(3) Flexibility in time, content, and methods

This principle complements the design principle “Design a methodological framework” in [8] since it explicitly asks for more flexibility in adapting the project objectives and methods during phase A, but also in phase B. It correlates with a different kind of research funding process though that would allow for timely flexibility.

Throughout the TDR process, but especially in phase A, the formation of a common research object and the framing of it are dynamic and may evolve over time. The better the scientists and practitioners understand the case, the TDR process, and each other, the more concrete and stable phase B will be. However, the team needs to show flexibility in content, methods and time throughout the TDR process, especially in phase A, to accommodate the growing knowledge and experience gained during the TD partnership. This demands a TDR skillset of the team for the use of multiple TDR methods, and patience and flexibility in time regarding the possible delays in TD integration (phase C). As confirmed by several studies, research that is more interactive and participatory requires much more time and flexibility than more traditional approaches [13,30,45]. Funding schemes should factor in this flexibility.

5.3. Phase B. The Production of New Knowledge

(4) Acceptance of process vs. project results

This principle complements the framework in both phases B and C. By being more specific regarding the satisfaction with at least the project process and partnership, if not with the results, it adds to the principles “Apply and adjust integrative research methods and transdisciplinary settings for knowledge generation and integration” in phase B, and “Realize two-dimensional integration”, “Targeted products” and “Evaluate scientific and societal impact” in C. As described in the case study reflection, despite a sound and successful TDR process, the nature of TDR may encompass a delay or even failure of TD knowledge generation in phase B, and TD integration in phase C. The project results may not be implemented within the project duration or at all due to, for example, cultural barriers or economic limitations—as experienced in cases 1 and 2. This type of delay and failure should be accepted as a normal element of TDR and, if it occurs, should not risk the overall evaluation (success) of the TD project. The sheer two-dimensional engagement in the TDR process and its outcomes should be seen as a success in itself, even if the targeted ‘product’ of the research, e.g. deriving and implementing policy advice, could not be achieved within the project duration. The practitioners may still be satisfied with the process and the benefits gained from the partnership of the TDR project in addressing their problem(s) framed in the beginning. The project outcomes do not necessarily have to solve the problem but at least provide a pathway indicating how solutions can be found. It is questionable whether practitioners should always be satisfied with the project results—what if local stakeholders had project aims with negative regional or global effects? I thus propose to focus on the project process instead of the results as the main measure of success.

5.4. Phase C. Transdisciplinary Integration

(5) Public outreach—science communication for a larger public audience

This principle complements the framework with its principles “Targeted products” and “Evaluate scientific and societal impact” by explicitly focusing on public and viral media outreach for a broader audience.

Although peer-reviewed scientific papers are required to ensure the overall academic quality and scientific rigor of the project outreach, too little emphasis has been put on communication with involved professionals and the broader public, who cannot be reached by academic papers. Popular science communication needs to be considered more in TDR dissemination, and one indicator of success here could be the number of people reached rather than the impact factor of a journal. One example of media which are growing in importance are short movies and photographs, which can be used to disseminate scientific messages of a complex nature to many tens of thousands of people, for instance, through TED talks (www.ted.com), YouTube, photo reports such as those featured in National Geographic, or even public media formats at large science conferences (e.g., [46]), and public science blogs—truly important when disseminating a message that is of relevance to society. Popular science disseminated via online communication platforms can easily and successfully be produced in terms of outreach and with no additional cost may go viral if it is of public interest. TD sustainability research projects should involve a stronger popular science communication component, and this importance should be acknowledged in the setup of the project team, the evaluation of personal skills of the research team, and the funding review system: in addition to impact factor ratings, a factor of success in TD sustainability research should be high-quality, popular outreach to a larger group of people, potentially spreading virally, including those outside of academia who are affected by the problem addressed by the project.

(6) Transition to a follow-up project partnership

A demand- or problem-driven flow from a finished project to a new one has been identified as a factor of success in the cases of this study. Thus, if such a transition to a follow-up project partnership occurs, the phases A and B are more likely to be successful. A smooth transition to a follow-up project requires that many of the above-described success factors are present, most importantly the TD skillset and flexibility, but quickly available transition funding based on the success of the main project in phases B and C is required as well.

6. Conclusions

What is success in transdisciplinary research (TDR)? This paper contributes to the discussion of defining and evaluating success in TDR by testing the design principles of the widely applied framework of [8] with the evaluative analysis of five TDR cases. My approach is to apply the TDR framework with its design principles to describe and analyze the five cases. Where the case study experience did not match the principles in [8], I explain their relevance for TDR based on the research teams’ experiences, and propose to add those complementary design principles to the tested framework.

Current institutional settings, in specific funding schemes, academic reward systems and the training of scientists entail barriers for successful TDR, especially for the early pre-project phase and the transition phase, and advancements here are urgently needed. The reflexive empirical approach I have taken in this paper offers a key step towards removing these barriers, because the findings demonstrate how conceptual evaluation frameworks can be used as applied evaluation frameworks. The derived complementing design principles are added in point to the areas where different contexts and cases will inform the best options and approaches for measurement and operationalization.

Overall, the general applicability of the design principles of [8] to evaluate the success of finished TDR and to design successful new TDR can be confirmed for the five cases. The majority of the design principles were found to match with the experience of the five cases. Some challenges in the design of TDR as expressed in the framework [8] were not encountered in the five cases, though, especially in phase A, formation of a common research object. This is mainly due to the five cases’ origin from society, where a common object had already existed before the project start, per definition an origin “from” society—where we, the TDR team, met an active demand. This confirms the importance of an early pre-project phase where a process of joint understanding between practitioners and scientists is supported (and funded) to develop the project demand-driven and not researcher-driven. TDR projects should ideally go with the flow of existing demand and real societal problems. The TDR skillset of the research team has shown to be of critical relevance during all phases of the evaluated projects, and its inclusion and application in the framework [8] should be much emphasized.

In addition to the validation of the overall usability of the framework [8] with its design principles, six principles derived from identified criteria for defining success are proposed to complement the existing framework. Principle number 1 is A project initiation based on the demand from society, guided by the flow of problem-driven research, and number 2 is Quick initiation funding to allow for the development of a TDR team. Sufficient time (and thus early funding) are critical for designing successful TDR in the earliest part of phase A; if scientists initiate TDR by “finding partners from practice”, the integration in phase A requires more effort as a crucial and challenging step in the TDR process. A guarantee of success cannot be given: as case study experiences show, during the TDR process the different mental models and different approaches of practitioners and scientists co-develop, and mental or cultural barriers may thus be discovered only at an advanced project stage 3. Flexibility in time, objectives and methods throughout the research process and 4. Acceptance of process versus product results have shown to be important principles for designing successful TDR. The success of TDR entails public outreach, and the stronger inclusion of public science communication like social media, in addition to academic papers, is suggested as complementing design principle number 5. New projects evolving from existing ones, which may proof that practitioners were satisfied and that the TDR team tapped into a related societal problem initiated from practice—the Transition to a follow-up project I propose as complementing principle number 6. Such a follow-up project may support the smooth transition from phase A to phase B. Success of TDR can be achieved without requiring further research, but if a finished project funnels into a new project, this transition will have proven the success of the previous project.

Further case studies should test the general applicability and completeness of these complementing TDR design principles. Concrete propositions on how to operationalize and measure them should be formulated.