Beyond Disciplinary Constraints: Designing Transdisciplinary Research and Collaboration in Real-World Laboratories

Abstract

Topics related to sustainable development are at the forefront of current local and global debates, involving social, economic, and political perspectives. To address these complex and interconnected challenges, it is necessary to think and act beyond disciplinary and field-specific boundaries. Participating in project work, research, experiments, and designing social transformation processes are at the core of transdisciplinary research and methods, such as Real-World Laboratories. This means that all participants are involved in co-designing, co-producing, and co-disseminating the project from the beginning. In these participatory processes, a systematic evaluation and evidence of the transdisciplinary process is often difficult to show even though it would be important to improve scalability and the transferability of the process. Using the example of the Tiny Fop Mob, a Real-World Laboratory on wheels, we wanted to understand how to evaluate transdisciplinary research thinking within the project team and demonstrate how systematic evaluation is crucial in participatory research. Findings show that there is a need to reflect on context-specific and team-oriented strategies during the planning and the beginning of the project.

1. Introduction

Only change is constant. This old wisdom is as relevant today as it has ever been. We are in a time in which change is happening at a rapid pace. It is becoming increasingly difficult to keep up with technological developments, demographic change is drastically altering our society in the coming years, pandemics and wars are showing how volatile and uncontrolled the whole society is, and finally, we are amid a climate crisis. We are now in a moment of “Great Unsettling” [1], where volatility, disintegration, insecurity, inequality, degradations, and dislocation are intensified and threatening familiar life-worlds. Scholars are speaking of multiple and interrelated crises, where a profound change is needed that is socially and economically desirable within planetary boundaries [2,3,4,5], speaking therefore of a great transformation in terms of Karl Polanyi [6]. It is, therefore, a matter of finding ways to shape and design the future. As Edgar Morin pointed out, the high complexity of these multiple crises asks for complex solutions putting together practical, scientific, and humanistic culture [7,8]. Therefore, solutions to problems should be developed together between different actors and at different levels: science, politics, economy, and society [9].

The topics of sustainable development, as defined in the Brundtland Report [10], p. 1 “as development that meets the needs of the present without compromising the ability of future generations to meet their own needs” and the following definition of the 17 Sustainable Development Goals by the UN, are at the core of current local and global debates at a social, economic, and political level to seek to achieve a balance between economic growth, social equity, and environmental protection. Being these multi-layered, complex, and interdependent challenges, it has been recognized that it is necessary to think and act beyond disciplinary and field-specific boundaries and to create a societal pact between science and society, where society has an active role, as argued in the German Flagship report “World in transition: a social contract for sustainability” [11]. This asks to change the conventional mode of knowledge production of the scientific system, creating a more embedded relationship between science and society and the societal relevance of science [12]. Therefore, science is moving forward to work beyond the disciplinary boundaries, speaking about multi-, inter- and transdisciplinarity. Multidisciplinarity involves parallel work by different disciplines on a common theme, with no shared problem definition or methodology, while interdisciplinarity, on the other hand, surpasses disciplinary boundaries by cooperatively integrating definitions and methods [13]. Transdisciplinarity takes integration further by involving non-scientific actors and addressing social problems through research. Transdisciplinarity fosters a transformative mindset to discover sustainable solutions in a dynamic global context with evolving cultural, technical, and economic imperatives.

Recognizing this need, for 15–20 years, the transdisciplinary research approach, putting together practical and scientific knowledge to jointly search for solutions and initiate change processes, is becoming increasingly important in scientific discourse [14,15,16]. The transdisciplinary research approach requires science to be open to problems of everyday life and to integrate non-scientific actors into the research process. Participation in project work, research, experiments, and designing social transformation processes are at the core of transdisciplinary research. All participants are involved in the co-design, co-production, and co-dissemination of the project from the beginning [17,18,19]. Following Bergmann [20], transdisciplinary research is made at the interface between society and science and is aimed at researching, transforming, and solving social problems by making problems and social actors a key point of reference of that research. In such a context, hybrid societal problems are solved in a transdisciplinary process. The aim is to achieve a transformation following two distinct epistemological principles, the path of practice (given by non-scientific actors) integrated with the path of science [20]. Transdisciplinary research wants to (a) grasp the complexity of problems; (b) consider the diversity of life-world problems and their scientific perception; (c) link abstract and case-specific knowledge; and (d) develop knowledge and practices promoting the common good. In that sense, the main objective is to provide descriptive, normative, and practice-oriented knowledge opening science to (a) lifeworld problems, (b) the integration of non-scientific actors, and (c) the explicitly normative treatment of its topics [20], p. 10. Transdisciplinary research works, therefore, in an interdisciplinary way and is understood as an elaborate research process that is designed together with non-scientific actors [20].

Thus, transdisciplinary research is mainly inspired by a practical need and consequently often takes the form of single case studies. These are initially treated only in isolation, often lacking the capacity for the difficult comparative evaluation of different cases. Nevertheless, as already observed by Mittelstraß in 2005, transdisciplinarity is neither a theory nor a methodology in a strict sense, but rather an organizational principle of research [21]. This organizational principle of research is used in many participatory, transformative, collaborative, and intervention-oriented research approaches, such as Real-World Laboratories, Living Labs, and other experimental approaches.

Real-World Laboratories, Living Labs, and Urban Living Labs

Real-World Laboratories (RWLs) are considered a permanent transdisciplinary structure offering enormous potential for knowledge integration across different single case studies [22,23,24]. RWLs are a possible format to foster transdisciplinary research. The scientific process in RWLs integrates different perspectives from science and practice [22]. In recent years, RWLs have increasingly gained importance. The region of Baden-Württemberg in Germany has been supporting RWLs since 2015 with important funding. The “Network of Reallabs of Sustainability (NRN)” (https://www.reallabor-netzwerk.de, accessed on 13 February 2023) consists of over 100 RWLs and organizations in Germany and beyond. There are many different definitions and interpretations of what RWLs are. The “Network RWLs for Sustainability” considers itself as a platform and network of RWLs to develop knowledge for sustainable development and to initiate concrete processes, seeing itself as part of a transformative research landscape. The definition used by this network is, “A real laboratory indicates a transdisciplinary research and development organization. Science and society work together on sustainable solutions. Universities, municipalities, NGOs, companies, government institutions, and associations join forces in RWLs under the guiding principle of sustainable development as pioneers of change (…). RWLs experiment, develop, test, and explore new things. The main goals of RWLs are to initiate transformation processes in a participatory and cooperative manner and to foster scientific and social learning processes.” (https://www.reallabor-netzwerk.de/zentrale-begriffe/was-ist-ein-reallabor/, accessed on 13 February 2023). Hence, RWLs are places where social change processes and sustainable, socially relevant, and practical solutions are discussed and designed in an exchange between science, politics, business, and the civil population. One of the most common definitions of an RWL is that of Uwe Schneidewind, who defines RWL as a place “[…] in which researchers carry out interventions in the sense of ‘real experiments’ to learn about social dynamics and processes” [25], p. 3.

Thus, RWLs provide a space to conduct so-called action or intervention research [26] or real experiments [25]. Action Research has a long tradition in social sciences. Social psychologist Kurt Lewin introduced the concept of action research in the 1940s [27]. Over the years, action research has become known by many other names and has been used in different disciplines, i.e., participatory research, intervention research, and experimental research. The key characteristic is of course the empirical and practical implication, where researchers and participants work together on common solutions in an iterative way of planning, acting, observing, and reflecting [27]. The aim is to collaborate to foster change in a system in a desirable way, where co-learning and co-decision processes are key features for success. Parodi et al. [26] state that RWLs can be understood as a form of institutionalized action or intervention research. This is the same for other approaches using action research as a frame for their action or participatory research.

Moreover, participatory action research is often associated with citizen science, which has become one of the most important research pillars for the European Union. Citizen science wants to involve the public (policymakers and society as a whole) with science in the different stages of the research process [28], but this is a much more open definition than for RWLs, living labs, or urban labs. These last ones have some characteristics in common. All three approaches can be considered as part of the so-called “experimental turn” in social science [29] and more specifically in sustainability studies. In governance studies, the experimental turn in social sciences refers not only to the participation but also to the experimental characteristic of the approach [30]. This means that the main objective is that the interventions are considered experiments by which citizens and other actors involved learn and are empowered. This is of course different from experiments in natural science or controlled environments. The social scientific interest is not the lab findings per se, but the usefulness and the characteristics of the possible change in the application at a real-world scale [30].

RWLs and Living Labs are very similar but pursue different focuses. RWLs focus mainly on eco-social transformation with a strong connection to sustainability, Living Labs aims to take more of a user-driven approach to foster innovation [31,32,33]. Their focus lies on testing innovations in the form of user co-creation or experimentation of innovations and prototypes in real time in the frame of public–private partnerships. The key point is that they are a sort of open innovation, as innovations are tested and developed in real-life contexts with stakeholders and end-users. The term Living Lab was introduced in the 1990s by Bajgier et al. [34] and then mainly developed in different projects of MIT around 2000. The main idea was to take innovation processes out of scientific labs into real-world settings [35].

Further development of the Living Labs is Urban Living Labs, much more similar to RWLS, which have become well known in the European context, also because they have been financially supported and introduced by policy measures of the European Commission. The network of European Living Labs (EnOLL), funded in 2006 by the Finnish Presidency of the Council of The European Union, is an internationally recognized non-profit association. They define living labs as “real-life test and experimentation environments that foster co-creation and open innovation among the main actors of the Quadruple Helix Model, namely: Citizens, Government, Industry, and Academia. (…) Living Labs are open innovation ecosystems in real-life environments using iterative feedback processes throughout a lifecycle approach of an innovation to create sustainable impact” (https://enoll.org/about-us/, accessed on 2 March 2023). The first phase in early 2000 counted 20 Living Labs in 15 EU member states with now more than 440 living labs all over Europe and beyond. Their main goal is to produce generalizable knowledge that can also be ideally used in other contexts [29]. Because of this context of innovation and experimentation, it is difficult to find any consensus of definition; therefore, it is vague and is creating confusion with other concepts, as with RWLs. Urban living labs are spatially limited to urban space, even though there are no defined boundaries. Some categories of Urban Living Labs are that they are a new form of collaboration to design, test, and learn from social and technical innovation in real time, where end users are enrolled in the innovation process and the knowledge transfer is organized by the support international networks [36].

RWLs and ULL are very similar, even though RWLs are more part of the German scientific discourse and focus more on sustainability discourses than on innovation and are not only limited to urban spaces. Both are part of a more general idea of research approach towards experimental and transdisciplinary research. RWLs have fewer clear objectives and focus more on the process of co-design and co-production of knowledge in a participatory and self-reflecting learning process [23]. Finally, these concepts are interconnected and share common principles. Transdisciplinary research often employs participatory approaches to engage diverse stakeholders in the research process, ensuring relevance and applicability. RWLs, Living Labs, and ULL provide structured environments for transdisciplinary research and participatory engagement, enabling the co-creation, testing, and implementation of sustainable solutions.

What all the different approaches in the field of transdisciplinary approach have in common is what some social scientists call an “experimental turn” [29]. From a methodological point of view, there has been a peak of social experimentation earlier, in early American Sociology, in the Chicago School [30]. These “experiments” in a non-controlled environment and the “real life world”, are continuously in a dynamic process and may change from situation to situation. This is one of the main common problems. It is impossible to generate universal knowledge that can be transferred identically to other situations because of the complexity of the inter- and transdisciplinary approach. Nevertheless, to overcome these issues, there are growing efforts to define characteristics [22,29,37,38,39] as well as evaluation criteria [38,40,41], but there are still no common definitions or defined methods for how to proceed in such projects.

Evaluating transdisciplinary research is complex due to its collaborative and interdisciplinary nature, but also because of the challenges and implications of power dynamics among the project team, the stakeholders, and the involved actors. Another issue is the difficulty of measuring the long-term impact that RWLs are producing in a specific context, which brings us to the next point: context-specific dependency. Therefore, having a transdisciplinary research strategy or frame on how to govern the process would contribute to the advancement of evaluation and impact assessment in transdisciplinary research, enabling a more robust understanding of the effectiveness, relevance, and transformative potential of transdisciplinary approaches to complex problem solving.

Starting from the case study of the RWL, we want to answer the following research question: How to evaluate and assess contextually dependent impacts of transdisciplinary research and thinking within the project team in RWLs? This research question aims to explore the effectiveness and suitability of the chosen evaluation methodology in capturing the transformative impacts of transdisciplinary research. It recognizes the need for evaluation methods that go beyond short-term outcomes and address the long-term and systemic changes resulting from transdisciplinary interventions within the project team and the community. By focusing on the context-specificity and transformative potential of evaluation methodologies, this research question seeks to contribute to the development of robust evaluation frameworks tailored to transdisciplinary research and RWLs and their impact on complex societal issues. The main challenge that we address is typical of RWL is its cyclical and process-oriented research approach, which needs to constantly be adapted to the needs of the actors that are not predictable from the beginning. Researchers have, therefore, additional and different roles, and the interventions leave traces that are not measurable in the long term. We agree that the transdisciplinary team has a crucial role in the process [42]; therefore, there was a special focus on the collaboration within the involved team.

2. Case Study: The RWL Tiny FOP MOB

The RWL Tiny Fop Mob project serves as a case study of how the governance of the transdisciplinary research within the transdisciplinary team has been organized and evaluated during the project. The overall project aim was to plan, realize, and use a mobile laboratory made exclusively of natural and regenerative materials to provide a tangible example of a sustainable building. The CO₂-negative space on wheels was used as an RWL on wheels to promote and initiate sustainable transformation processes within companies and communities in rural areas. The Tiny Fop Mob (German: Kleines Forschungs- und Praxismobil) wanted to encourage sustainable transformation processes through two main steps: (1) the construction of a CO₂-negative prototype using a combination of wood, hemp, and coal; (2) the prototype itself serves as an RWL in which participatory workshops, research, design interventions, and events were organized to raise awareness of entrepreneurs and the civil society regarding the relevance of sustainability and to initiate a socio-ecological transformation process.

The project is based on two pillars: (1) the applied level to plan and build the prototype out of the innovative combination of hemp, wood, and coal; and (2) the awareness level to show companies and civil society how sustainability can be applied in practice (by building and discussing the CO₂-neutral prototype, offering workshops, etc.). Between July 2021 and June 2022, the Tiny Fop Mob RWL was on the road in five different municipalities of the Vinschgau Valley, a valley of the Autonomous Province of Bozen-Bolzano, a German-speaking region in Northern Italy. During this trip, numerous experiments and surveys were conducted, workshops were held, business evenings and other participation formats were offered. In this way, the project contributed to raising awareness and providing information in the field of sustainable development and sustainable construction and housing. The participatory research approach focused, therefore, on two distinct levels: on one hand, involving the whole project team in a transdisciplinary and participatory process, and on the other hand, involving the communities via the mobile RWL.

Beside the planning and the construction of the RWL by the two business partners of the project (the producer of the bricks made from hemp and lime and the carpenter), the following scientific research has been carried out by different research institutions: analysis and governance of the transdisciplinary research process; real-lab experiments and workshops; survey and analysis of the sustainability awareness of visitors; test of the wood and hemp walls in the Façade System Interactions Lab of Eurac Research; evaluation of the comfort (air quality, room temperature, etc.); experimental tests of visitors’ perceptions; and economic sustainability and a LCA (life cycle analysis). There was, therefore, a variety of scientific disciplines and backgrounds in the project, mainly without experience in transdisciplinary research projects.

3. Method

The case study was a project initiated by a team interested in building a mobile RWL and fostering sustainability in the building sector and rural areas. Therefore, together with a producer of hemp bricks, who is considered a pioneer in his sector, a carpenter working with local wood, some engineers from two research institutes specialized in different fields (sustainability, comfort, and facades) in the building sector, we built a multidisciplinary team to begin a transdisciplinary RWL. We, the initiators, coming from social sciences and already experienced in inter- and transdisciplinary participatory projects, defined a research design that wanted to foster, test, and evaluate the transdisciplinary process from the beginning.

The core team of the project was made up of three social scientists, three engineers, and two practitioners (business owners). The main role of the core team was on one hand to make the decisions and to co-design the project, i.e., the design of the Tiny Fop Mob; the decision in which villages the rolling RWL will stop; the kind of interventions, activities, and experiments that will take place; the involvement of the stakeholders and citizens; the different type of scientific measurements (such as comfort analysis, life cycle assessment, etc.); the communication strategy; etc. On the other hand, the core team was asked to evaluate with their team the transdisciplinary process using a questionnaire and by taking part in regular self-reflective workshops to redefine the process.

3.1. The Building of the RWL

The first important part of the project was the building of the RWL itself; the planning and the building of the mobile Tiny Fop Mob using the combination of hemp bricks and wood was mainly led by the two practitioners in the project team, even though the planning phase was carried out in two transdisciplinary workshops with the core team. These were the beginning of the common findings of solutions and exchanges between the involved partners. The construction process took around six months and involved conceptual sessions and experiments with hemp and wood materials. The objective was to test and implement a wall structure made entirely of natural resources by combining hemp and wood. The result was a prototype on wheels (the Tiny Fop Mob), representing a small, CO₂-negative space that served as a RWL within the project. The planning phase involved creating initial sketches to develop a clear vision of the laboratory. The project team aimed to create a mobile laboratory with sufficient space for work, workshops, and exhibitions. As a result, the decision was made to build the laboratory on a truck trailer. The Tiny FOP MOB measured 10 m in length, with an additional terrace of 3.40 m, a width of 2.48 m, and a height of 2.70 m. It comprised an entrance area, a terrace, and an interior equipped with furniture and lighting. Due to pandemic restrictions, the interior space was primarily used as an information area, accommodating only a limited number of people at a time. Events and workshops were conducted outside or in community-provided halls during unfavorable weather conditions. In terms of material selection, sustainability was a key consideration. Wood, such as spruce, larch, and stone pine, was used for wall frames, flooring, and furniture. Hemp was utilized for bricks, plaster, insulation, and soundproofing. Additional materials, including a vapor barrier and steel sheet, were incorporated to ensure the mobile laboratory’s functionality under various environmental conditions. Overall, the Tiny FOP MOB project aimed to showcase the possibilities of sustainable construction and foster community involvement through the real-world laboratory.

3.2. The Journey of the Mobile RWL

The second important part was the journey of the RWL in the rural region of the Vinschgau Valley. During the journey of the mobile RWL throughout the Vinschgau region from July 2021 to June 2022, numerous experiments, surveys, workshops, company evenings, and other discussion formats were conducted. The Vinschgau region was chosen for two main reasons. First, the region is characterized by a high number of grassroots initiatives and innovative projects driven by committed and visionary individuals. Sustainable development has been a central topic addressed in various ways by organizations, associations, businesses, and individuals in the region for decades. Second, the Vinschgau was selected as the project region because the two participating companies are located there. Before starting the journey, we conducted 12 qualitative interviews with stakeholders and pioneers in the field of sustainability to understand the relevant topics in the region, to identify possible pilot municipalities, and to make an actor-mapping of existing activities and actors in the region. In consultation with the core group, five pilot municipalities were selected for the RWL to visit. Factors such as population size, economic structure, size of the artisanal zone, and the presence of potential cooperation partners and projects were considered. The goal was to reach as many people as possible and cover the entire Vinschgau region, taking into consideration the spatial distances between the municipalities. The chosen municipalities were Graun im Vinschgau, Mals im Vinschgau, Prad am Stilfserjoch, Schlanders, and Latsch. The real-world laboratory spent approximately two weeks in each of these five municipalities. The questions that were aimed to be discussed together with the citizens were about the average land consumption per person, the materials predominantly used in construction, the energy sources tapped into, the amount of CO₂ emitted per capita through building and housing, and the sustainability of supply chains. Ultimately, the overarching question was how to make building and housing more sustainable.

A key objective of the research project was to engage in discussions with companies, experts, and interested citizens to address these and other issues related to sustainable building and housing. The core team decided to dedicate each municipality to one of the so-called sustainable building life cycles. A sustainable building life cycle encompasses all phases that a building goes through, from raw material procurement to demolition, disposal, or recycling. It can be divided into five phases: (1) raw material extraction (Schlanders); (2) planning phase (Latsch); (3) construction phase (Graun im Vinschgau); (4) use, repurposing, modernization, and maintenance (Prad am Stilfserjoch); and (5) demolition, reuse, or dismantling (Mals im Vinschgau). The results of the discussions were summarized finally in ten measures for sustainable building and housing by the core team. In each municipality, we foresaw an opening event, a public workshop, and a business talk. Each of these different events foresees the invitation of an expert to hold a short input speech and then an interactive and participative discussion with the citizens. At each event, we invited stakeholders, citizens, and local businesses. On average, about 15 persons participated in the events, in total 298 persons participated in the 20 events/workshops, and additionally, 805 persons visited the Tiny Fop Mob during opening hours.

3.3. The Governance and the Evaluation of the Transdisciplinary Process

The third important part of the project was the so-called “governance of the transdisciplinary process”. As already mentioned, the core team was composed of sociologists, political scientists, sustainability experts, communication experts, engineering scientists, practitioners, and craftsmen. As observed by Huning et al. [42], the team constellation is an important aspect of the success of the RWL. First, the constellation depends on the research activities and objectives. Second, there are different types of involvement of the actors: the core team members, engaged stakeholders, and loosely connected stakeholders that only participate occasionally. Through the collaboration with stakeholders in the municipalities and an innovation hub of the region, engaged stakeholders, such as the local innovation hub BASIS Vinschgau, had the role of information brokers and engaging citizens. Loosely connected stakeholders were also present in the project, mainly if they had any personal interest in the theme. As we wanted to focus the transdisciplinary process not only on the frame of the RWL with citizens and stakeholders as the primary research population, we decided to ask the whole project team (the core team and other collaborators) to take part in an internal evaluation. This was important because one of the main shortcomings of RWL and other participatory formats observed in the literature is the missing systematic evaluation [29,37,38,39]. The internal evaluation of the project consisted of two distinct purposes. On one hand, we wanted the entire team to work towards a common transdisciplinary output throughout the entire process, even though each member conducted different analyses and had different tasks. We did not want to lose the transdisciplinary perspective on the entire project, in terms of finding common solutions to the problem from both scientific and practical perspectives. Regular workshops (every 4 to 6 weeks) were organized to jointly define the success criteria for the entire team, ensuring that for each of the different perspectives the project was successful. This was important, as we know, since often in project teams one of the main challenges is to overcome different expectations and objectives of the involved actors [23]. This required reflecting on the original project idea and the project implementation (construction and journey of the real-world laboratory) and adapting the ideas and goals of individual components to overarching, collectively developed concepts. To strive for transparency and minimize bias, following Max Weber’s concept of axiological objectivity, we recognized our subjective values in research and openly acknowledged personal perspectives that might influence the work, engaging in periodical reflection to identify potential bias sources. While complete value detachment may be unattainable, we aimed to minimize bias via diverse methods, triangulation, and transparent communication during the whole process and the workshops. A balanced approach is sought, wherein subjective values are acknowledged but rigorously managed, ensuring credibility and objectivity in the research process and findings.

Therefore, since the beginning of the project, to focus on common objectives, we decided to define the success criteria based on studies on other RWLs and on the project aims. These success criteria have been monitored, discussed, and reflected weekly by the project coordinators and periodically during each partner meeting and workshop (every 4–6 weeks) to recognize failures and react. We co-designed the success criteria with partners using the following participatory approach. At the beginning of the project, the project coordinators, who took care of the transdisciplinary process, extracted the different aims and objectives defined in the project proposal. In the first phase of the project, during a workshop with the project partners, we defined in a multidisciplinary approach what the criteria are for each partner to define for themselves the success of the project. The second step of the workshop was to match and compare the aims and objectives of the project with the perspectives of the project partners and define common success criteria and how to measure it. We defined the success criteria that we decided to monitor during the process and re-discuss in a self-reflective way at the end of the project. This was necessary because the different partners joined the RWL with different motivations and expectations according to their own logic and interests. Engels and Walz [43] defined co-design strategies to integrate these multiple perspectives, which we considered in our success criteria (mutual learning between team members, creating ownership, creating and maintaining trustful relations, engaging with citizens, and giving room for change).

Moreover, to evaluate the process and to understand the integration of transdisciplinarity individually, we asked each project partner to respond to the questionnaire developed by Bergmann et al. [14,44]. This is a guide for the formative evaluation of research projects developed by the “Evalunet—Evaluation Network for Transdisciplinary Research”. The evaluation guide includes a catalog of described quality criteria for all project phases, ranging from conception, selection of research partners and stakeholders, financial considerations, project leadership, and knowledge integration to the implementation of research outcomes into products, publications, or consultancy services. The evaluation process is both formative and discursive because it facilitates learning processes and a shared understanding of project objectives and methods. After researching the literature, we decided to utilize this proven tool, which provided us the possibility to evaluate the involved actors, the structure of the research project, the project evaluation and methods, and the results and findings. The authors were members of the transdisciplinary project team and completed the questionnaire, just like the other project partners. Utilizing the questionnaire also facilitated additional transparency; the results were not anonymous but instead were employed for subsequent group discussions. The analysis of the results was conducted by scientific collaborators who were external to the core team and did not participate in questionnaire completion.

Each partner (mainly the core team) was asked to answer the questionnaire (two for the different engineering teams; two for the practice partner; four for the lead partner involving administrative staff) at the beginning, in the middle, and at the end of the project. The questionnaire was sent to the partners by e-mail; their written answers to the open-ended questions were analyzed using a qualitative and comparative approach. On the one hand, the results were represented numerically based on the agreement with the statements in the questionnaire. The answers “Yes” were recorded with the highest value (“3”), the answers “Partly” with the value “2”, and the answers which were answered with “No” with the value “1”. This was then aggregated into an average score to analyze the progression over time. In addition, the partners had the opportunity to comment on their evaluation in each case. These statements and the answers to the open-ended questions were systematically analyzed and compared in terms of content analysis.

During the whole process, we collected data from the project partners to monitor and evaluate the transdisciplinary process within the team and the project with the participants and citizens during the activities. In the presence of the RWL in the municipalities, we decided to have opening hours during weekdays when a researcher was present. At that time, the role of the present person was mainly to provide information to curious people, to invite them to the activities, and to take part in the research. We collected ethnographic data using field notes, informal interviews, and feedback sheets. Moreover, we developed a questionnaire including engineering, perception, and social aspects of comfort analysis, perception, and sustainability awareness. The visitors could voluntarily participate in this exploratory, non-representative survey; a total of 148 valid questionnaires were then evaluated by the different research teams.

4. Results

As explained in the section before, the definition of common success criteria throughout the project process and discussing them with all partners helped the project team to keep focused on the common objectives while focusing on each different part of the project. In the frame of this paper, the main issue was to show how we managed to measure if inter- and transdisciplinary thinking happened. The definition of ten success criteria in combination with the questionnaire allowed a systematic evaluation and self-reflection, and to overcome systemic problems related to different (subjective) perspectives, power relations, and negotiations among the partners [42]. The evaluation of the questionnaire helped to understand shortcomings in the project planning and execution that would be important to focus on during future projects. Through the discussion of the criteria and the evaluation guide, we could observe the differences between the beginning and the end of the project, both in the answers to the open-ended questions in the questionnaire and during the discussions in the workshops. For example, only at the end of the project, one of the partners stated that he now understood what the work package “transdisciplinary governance process” was intended for and added value because before he worked only in multidisciplinary teams. First, we will show the results of the questionnaire and then the success criteria.

4.1. The Questionnaire

The results of the questionnaire on the individual evaluation of the transdisciplinary process are shown in

Table 1. Results of the questionnaire—Average agreement of all project partners. 3 = yes; 2 = medium; 1 = no.

		Average Round 1	Average Round 2	Average Round 3
Sector (A) Actors and Project design	Existing competencies	2.60	2.80	2.40
	Current and relevant issue	3.00	3.00	3.00
	Methods used	2.40	2.60	2.20
	Planning of the project	2.40	2.00	2.00
	Involvement in the project processes	2.80	3.00	2.80
	Financial and organisational resources	2.20	2.20	2.20
Sector (B) Project Implementation	Communication and participation	3.00	3.00	2.60
	Connection of disciplines and practice	0.80	2.40	2.40
	Regular reflection and communication	3.00	3.00	2.80
Sector (C) Results, Products, Publications	Solution of the problems			1.80
	Achievement of project goals			3.00
	Appropriate project results			1.80
	Matching and communication of results to target groups			1.80
	Generalization of research results			2.00
	Added value of the transdisciplinary approach			2.40

. Each partner involved could answer if they agreed with the statements or not and could then add their comments. In Table 1, there is an average of all values of the project partners, and the agreement ranges from 1 (no agreement) to 3 (full agreement). The comments and the discussion in the meetings were important to understand the answers. For instance, if the first question “existing competencies” were evaluated positively in the first round, it resulted in being perceived negatively that nobody in the team has experience with Tiny Houses even though this was not part of the project’s aim to focus on Tiny Houses. Because of the tininess of the prototype, much of the results of the measures (CO₂ absorption, LCA, etc.) could not be used because the volume of the room was too small and not at a real scale for a living room, this was an important discussion point between the different expectations of business and academics. Another main problem in the project was that there was not enough time: RWL needs to be planned as a long-term project and with a follow-up already planned before the project ends. The “connection of the disciplines and practice” was evaluated as negative at the beginning (average 0.80) but, in the end, resulted in being perceived as a positive experience (average 2.40). Also, the “added value of transdisciplinary approach” was evaluated positively at the end of the project (average 2.40) even though in the comments the researchers were still missing a common methodology for transdisciplinary research.

The results of the analysis of the open-ended questions helped to understand and interpret the answers and served for the discussion in the workshop. In general, all partners had concerns about timing and workload: there is a need for more time and human resources because of the intensive workload of transdisciplinary projects. This kind of funded project often demands too much bureaucracy that impedes the focus on the main goals of the projects. The involved practitioners needed much more coordination, partly because of their lack of experience in scientific projects. The engineers and practitioners would have preferred a structured method, especially because they had little experience in process- and participation-oriented projects and so had some difficulties with the ask for flexibility and adaptability during the process. In general, for the involved partners, it was a novelty to work on a transdisciplinary project because the high demand for reflective and participatory involvement in the project was new for them, as stated in this quotation: “I believe there is not yet a methodology to combine all the contributions completely. I think there is still work to be done on this aspect, but I think this is because it is new for everyone involved”.

4.2. The Success Criteria

The primary objective of this paper is to elucidate our approach to assessing the occurrence of inter- and transdisciplinary thinking using the questionnaire and the success criteria. These criteria were developed through reflection workshops involving the partners, serving as predefined benchmarks or standards against which the outcomes of a project or process can be measured. By directly relating these criteria to the research question—how to assess the contextually dependent impacts of transdisciplinary research and thinking within project teams in Real-World Laboratories (RWLs)—we endeavor to demonstrate how these criteria, as indicators, shed light on the multifaceted manifestations of transdisciplinary thinking within real-world contexts. They offer a predefined set of qualities, attributes, or achievements that are considered indicative of success, at all different levels of the project. In the context of measuring inter- and transdisciplinary thinking, one success criterion acted as an observable sign that indicates the presence and effectiveness of such thinking within a project or team. For instance, part of success criteria 1 (inter- and transdisciplinary thinking) was “Effective collaboration among team members from diverse disciplines”, which can be seen as an indicator of inter- and transdisciplinary thinking happening within the project. Similarly, the criterion “Integration of insights from various fields into problem-solving approaches” can also indicate the presence of transdisciplinary thinking. Both of those criteria were observed by all the participants. These criteria provide a tangible way to understand the extent to which inter- and transdisciplinary thinking has manifested, making it possible to measure and report on the degree of success in fostering such thinking within a given context.

It was important to facilitate, support, and analyze the whole transdisciplinary process through an internal weekly discussion and evaluation by the project coordinators who oversaw the governance of the transdisciplinary process and at the same time more expert in the field. The goal of the transdisciplinary process governance evaluation performed weekly by the project coordination team was to bridge the gap between different research approaches on the one hand and business practice on the other hand, capture different perspectives and viewpoints, recognize existing challenges, and turn them into opportunities, and finally, gain new understandings of how to design and execute a transdisciplinary research process. It was, therefore, important to conceptualize for each periodical partner meeting part of the workshop dedicated to the evaluation and reflection of the success criteria and the transdisciplinary process. The main activities at the beginning of the project were dedicated to involving the whole project team in this process: coming from practice, engineering, or other disciplines, they did not have any experience before in transdisciplinary research. First, it was important to gain and develop a common language due to different interpretations of technical terms to facilitate communication and promote mutual understanding and appreciation, develop and design new methodological approaches and techniques to solve problems, and finally recognize timely success and failures of knowledge integration. For example, the understanding of concepts, methods, and languages from different disciplines and practices was part of the transdisciplinary process and integration in terms of mutual learning. The following success criteria were outlined as an outcome of the project; some of them are similar, but they rely on the agreement of the project partners:

(1)

Inter- and transdisciplinary thinking and acting of the project partners;

(2)

Participation in civil society;

(3)

Creating added value for businesses;

(4)

Bringing science to rural areas;

(5)

Involvement of stakeholders;

(6)

Communication internally and externally;

(7)

Participative decision-making process in the project team;

(8)

Cross-sectional objectives of the project;

(9)

Publications and conferences in different disciplines;

(10)

Evaluation, self-reflection, and flexibility.

After introducing some success criteria from the project application from the beginning and then discussing and adding new criteria during the whole project duration, in the last partner meeting all the project partners agreed on those ten criteria. The sequence of the criteria did not represent a ranking of the criteria, as all of them were of equal importance. We would first explain the intent behind each criterion and then re-discuss them at the end and how they could be used and aggregated for further projects.

The first criterion (“Inter- and transdisciplinary thinking and acting of the project partner”) underlines the importance of bringing together researchers from different disciplines and people working in other sectors and civil society, including a continuous common reflection on the process by periodical meetings and evaluations to understand different perspectives and find solutions to problems. In practical terms, we wanted to observe if there was an effective collaboration among team members from diverse disciplines or an integration of insights from various fields into problem-solving approaches. The aim of these criteria where to keep in mind during the whole project that there are common objectives, even though each partner has different perspectives. One partner, for example, did not really engage in the process and pursued only his objectives in taking some technical measures and only sharing the outputs without discussing troubles nor participating or engaging with citizens.

The second criterion “Participation of civil society” means not only thinking about how to motivate civil society to participate in the process but also collecting data and information during the various experiments to analyze visitors’ attitudes, views, and opinions. Data were collected through feedback forms, questionnaires, or participatory observations. In the Tiny Fop Mob Project, about 1100 people visited the mobile RWL; 298 persons participated in the main activities; 30 companies were represented during the workshops; three workshops with middle schools and one workshop with students from the Faculty of Design and Arts were organized; and 148 questionnaires and feedback forms were collected. The workshops with the schools and the students were not planned at the beginning, which also shows an important impact of the territorial presence of the RWL and the planning flexibility. The results of the workshops with the citizens were integrated into the definition of specific policy measures that we defined at the end of the project.

It was important to create added value for all involved project partners (third criterion), both at a scientific and practical level, i.e., in this case, the involved companies. This means respecting all the different interests of the involved partners and integrating multiperspective views. For the companies, for example, it was important to realize an innovative product by exchanging ideas and know-how with academics and to have access to scientific results, giving them new inputs and innovative ideas. It is important to consider not only the sustainability aspect of transdisciplinary projects and the RWL but also to focus on the needs of the business in terms of their entrepreneurship and innovation-driven activities [45]. The exchange with other companies during the RWL created new business networks with academia and exchange knowledge with other companies in the main interested area.

As scientific actors are more often concentrated in urban areas around universities and innovation hubs, it was important to bring science to rural areas (fourth criterion), bridging the gap between innovative approaches and knowledge of the urban centers and rural areas. Therefore, it was necessary to facilitate communication and collaboration between different actors in both areas to exchange knowledge and strengthen connections and cooperation.

The criteria “involvement of stakeholders” from the beginning and even before the project is a warranty that the project is also accepted and supported locally. Having personal connections to the region or being introduced through the stakeholders can be helpful in gaining the trust of the population. This also helps to collaborate and exchange ideas with local partners and create new networks of collaboration. This was different from the involvement of citizens, the stakeholders had mainly a double role as “brokers” in spreading information and inviting citizens to participate. In such small rural areas, stakeholders have a key role in the success of the RWL.

One main activity to focus on during the project, which should, in our opinion, be one separate work package and, if possible, include an expert partner in the project, is the communication part. In addition to improving the image of the research and the project partners for dissemination purposes, the more the project is present in different types of media, the more the project partners can identify themselves with the project, and civil society is aware of what is happening. Communication inside and outside the project is one of the main channels to create trust among the project partners and the specific location.

The criteria “cross-sectional objectives of the project” and “publications and conferences in different disciplines” are mainly related to the research project. The cross-sectional objectives achieve the goals of sustainable development, gender equality, equal opportunity, and non-discrimination. The publications and project outputs are mainly important for the scientific partner, but a final report for a wider audience is, in our opinion, important to involve other non-scientific actors as well.

Finally, the last criterion is related to the transdisciplinary process itself: “evaluation, self-reflection, and flexibility”. This was also performed by the questionnaire and periodic meetings and self-reflections during the process. Flexibility is necessary in the case of RWL where the work is performed by and with persons because there is often a need to change programs and adapt them to different situations. The success criteria outlined within the RWL Tiny Fop Mob finally resulted in being very similar to those defined by Jahn and Keil in their analysis of 14 different RWLs [37].

As explained, these success criteria for the specific project were defined by the project partner to keep in mind the project requirements and as a self-reflection tool for the project’s success. The project coordinators in charge of the evaluation of the project analyzed the discussions during the workshops and partner meetings to understand how this tool helped to improve transdisciplinary thinking and asked for feedback from the partners at the end of the project. For the practitioners, even though at the beginning it was seemingly a waste of time, they understood better academic processes and could better understand some results that are only “academic”, in the sense that they did not have any relevance for their work. On the other hand, engineers understand that businesses expect a practical implication for their work and that they are not happy with scientific results if they do not match their expectations. This was the example of the outcome of the life-cycle assessment of the hemp bricks: the business wanted the product results to be the best for their interest; however, the engineer’s research showed scientific data that did not match the objectives of the business. Therefore, they needed to find a solution for how to produce reliable scientific data matching both objectives with new data collection and calculations. While engineers had easier ways to publish their results, for social scientists it was difficult to find common possibilities to publish due to different academic disciplines and fields. The role of the social scientists in the project was much more to coordinate and facilitate the process and organize the interventions of the RWL and less as “real” academics producing technical data for the business.

5. Discussion

Focusing on the participative and transformative character of RWLs, the main goal of the transdisciplinary integration is to better understand causes and impacts, to identify problems at an early stage, and to develop measures and solutions together with those who are affected. RWLs pursue the idea of conducting experiments together with civil society providers of social innovations. This means in general that the main tasks of the researcher are planning, designing, and developing intentional change for a specific eco-social transformation. This demands a lot of different skills, researcher have different roles and are in a continuously changing process asking for flexibility, but also the need to continuously adapt to the process, to communicate between the different involved actors, be open to new approaches, be in continuously collaboration with different actors, being constantly in a situation of adaptation and problem solving, and finally the ongoing risk of failure. This kind of project requires much more “extra-professional” skills than conventional research projects: this was quite new to some involved researchers in the projects, while for others, already with experience of other inter- and transdisciplinary research projects, it was easier to deal with these requirements. These skills are mainly acquired by “learning by doing”, but this means having enough time to focus on team building and constant self-reflecting work [42].

Academics can be [46] change agents (capacity building and implementation intervention); knowledge brokers mediating between different knowledge; reflective scientists providing scientific knowledge; self-reflecting scientists reflecting on their own positions and relations; and the process facilitator enabling transdisciplinary knowledge through constant encouragement of collaboration and communication. The team was composed of sociologists, political scientists, sustainability experts, communication experts, engineering scientists, practitioners, and craftsmen and each member had different roles in different moments. Moreover, those having more experience in inter- and transdisciplinary projects could switch and move between the different roles, while it was difficult for some involved academics to go beyond the role of “reflective scientist” and integrate into the transdisciplinary process, for example not being able to find solutions how to integrate measurement failures as part of the process or to try to find solutions to create usable results for the involved business.

In RWLs, transdisciplinarity and transformation toward sustainable development are at the core of the research process. This means that scientists and practitioners work together to develop the research and transformation agenda. They want to produce orientation and transformation knowledge identifying possibilities and limitations of decision-making processes and providing the means to implement concrete solutions and decisions [37]. Orientation knowledge identifies the possibilities and limitations of decision making. Transformation knowledge provides the means to implement concrete decisions in such a way that the intended social change processes run in the desired direction [37]. Thus, RWLs are related to learning and education. The RWL itself is both a place for learning and a social learning process. Researchers act and reflect together with social actors. This means, that there is an iterative process of collecting experiences, reflecting, and changing, to learn with and form each other in social exchange, but also that different societal issues are called to be solved or transformed by working with different stakeholders.

As seen during the process, another important topic is the location of the RWLs, but not only in terms of rural areas. Giving the possibility to have a space or a place to come together and, as shown in the Tiny Fop Mob, to go to the people is important to raise the participation and the interest of the population. Central requirements for this place are visibility, accessibility, and addressability [22]. Also, here the role of the scientists is different from other scientific projects. The principle is, that everyone in the project is considered an expert in their field, researchers also have the role of facilitators, designers, moderators, planners, and problem solvers when coming to specific locations, engaging with stakeholders, and acquiring knowledge about the specific context.

The terms experimentation and laboratory are often associated with the natural and technical sciences. Experiments carried out in RWLs are called “real experiments” [25]; however, there is an “uncertain outcome”. So, as Parodi et al. [26] observe, there is still a negative connotation to experimentations; therefore, they propose to instead speak of “transdisciplinary experiments”. These are characterized by the fact that the experimental setting itself, i.e., design, implementation, evaluation, and exploitation, is open to social participation and is therefore always depending on the process. Thus, the main common problem, in Living Labs or other forms of social participation processes, is the transferability of the results [47]. What differs is the goal or the objective. While Living Labs and Urban Living Labs aim at generating transferable solutions, RWLs focus on the participative and transformative character, which are process-oriented and process-related [29,39]. Therefore, working on long-term solutions and following evaluation and success strategies help to improve the possibility of creating transferrable solutions.

6. Conclusions

All the mentioned approaches above are transdisciplinary and have the aim to involve different actors in a process of social participation and transformation, with a limited impact in terms of transferability and impact evaluation. As already mentioned, transdisciplinarity is not a method but a process. This means that for researchers wanting to engage in such processes, there is a need to be open to other disciplines and practices and to handling possible conflicts and possible failures. Working with different disciplines, practices, and stakeholders require also strong communicational skills and an engagement in trust building. This needs intensive work in the field, but also involvement and openness. Communication is a central success factor: investing in communication is crucial for participation and learning. Moreover, the presence of a physical object, a construction, or an open space over a longer period is necessary to gain the interest and the trust of the population.

Some of the shortcomings of the project are that we focused mainly on the transdisciplinary governance and process within the project and then on the long-term evaluation and the societal impact. It would, therefore, be important to plan an evaluation in terms of the impact of the transdisciplinary process on the citizens and then plan an evaluation after the end of the project to see how the RWL created long-term results. Involving specifically businesses is also relatively new for this kind of project, as businesses often do not necessarily match with the normative idea of RWL. It would also be helpful for them to plan and evaluate their corporate social responsibility [48] and how this relates to the transdisciplinary purpose of RWL.

In the literature, other authors likewise define characteristics and criteria for the successful running of RWL that are very similar to those defined and observed in the Tiny Fop Mob project. Beercraft and Parodi [22] define seven characteristics of RWLs: research orientation, normative orientation towards sustainability, transdisciplinarity, transformativity, civil society orientation, long-term orientation, and laboratory character. Very similar to those, Schäpke et al. [39] identify five core characteristics analyzing their own research and other laboratories: contribution to transformation; experiments as core research method; transdisciplinarity as core research mode; long-term orientation, scalability, and transferability of results; and learning and reflexivity.

Although the success criteria served as an indicator for this specific project and there are similar outcomes in the literature, we recognized the importance of elevating the quality and practicality of our success criteria to be usable in future research projects. This involves multifaceted enhancements. There is a need for a comprehensive reflection on the criteria’s quality, how they conceptually relate to the context of transdisciplinary research within Real-World Laboratories (RWLs), and how to avoid redundancy. Another important critical point is the measurability; it is necessary to provide operationalization and quantification of each criterion. This will facilitate their effective utilization in evaluating the extent of transdisciplinary thinking in future projects. Furthermore, it will be important to emphasize how success criteria intricately align with the fundamental tenets of transdisciplinary research, promoting interdisciplinary collaboration, integration of diverse knowledge domains, practical relevance, and ethical considerations. Through these nuanced refinements, it will be possible to bolster the inherent value, practical utility, and contributions of the success criteria to the advancement of transdisciplinary research practices.

These approaches are not new; for instance, in the field of design, transformation design is emerging as a new discipline, positioning transformation and the related transdisciplinary approach at the core of the research agenda [49]. It is important to recognize that the solution is not only the action but the idea of drafting and designing change and transformation through social participation instead of top-down decisions, integrating experts, producers, consumers, planners, and citizens in an interconnected process of designing [50]. Therefore, there is no linear process or solution toward transformation; however, it is a dynamic process that needs a new way of organizing the approach to social sciences, sciences, and practice more broadly.

Finally, the fact that the success criteria defined during the project and those identified in the literature are very similar shows that there is the potential for transferability and common characteristics of the different RWL projects. The main challenge would be to focus on power dynamics that can hinder effective participation in the governance process; communication, which can result in misunderstandings, lack of clarity, and misalignment of goals and objectives; time constraints that can affect the quality of the transdisciplinary process; and finally, resource constraints that can affect the participation of project partners, especially those not having enough resources. The systematic evaluation of the project will help to address these challenges. But, as observed by Huning et al. [42], ambiguous roles, different strategies, and work with challenges within the team are at the core of the work in RWLs and should be taken into account from the beginning of the planning of an RWL, foremost for the core team in the project.