Evaluating Collaboration in a Translational Research Ecosystem

Abstract

A core challenge of a multidisciplinary and multi-organizational translational research program is to set up and promote collaboration between researchers, labs, and organizations. Although the literature has studied and provided guidelines for collaboration, little has been written on how to evaluate it in large research projects and in a practical way. This study aims to identify dimensions and barriers to evaluating and leveraging collaboration in a large translational research ecosystem related to developing phytotherapy-based cancer treatments. By applying the Collaboration Evaluation and Improvement Framework (CEIF), our paper adds value by developing a methodological design for evaluation, incorporating mixed data in a real research ecosystem. Empirical findings provide support for applying the assessment approach and show that a research project’s sustainability depends on several collaboration factors and barriers at the socio-technical, management, operational, and institutional levels. Research results provide valuable insights for managing and improving collaborative efforts in large research groups, by anticipating collaboration issues with actionable and opportune strategies that can enhance the planning process, ecosystem performance, sustainability, research outcomes, and the program’s overall success. As a result, monitoring governance, management, leadership, and social relationships throughout the different phases of a translational research program is crucial in assessing and promoting collaboration.

1. Introduction

Translational research bridges the gap between scientific discoveries and practical healthcare, comprising four phases: T1 (basic research), T2 (preclinical studies), T3 (clinical trials), and T4 (implementation and dissemination). The goal of translational research is to bridge the gap between basic science and clinical practice, with the ultimate aim of improving patient outcomes [1]. This has been particularly relevant in cancer research because of recent advances in our understanding of the molecular characterization of tumors and the tumor microenvironment, enabling the development of novel treatments [2]. According to Dolgos et al. (2016), by integrating basic science research with clinical development, translational medicine has helped to improve target understanding, patient subpopulation selection, and organizational learning. It has also led to advances in areas such as disease model validation, human cell- or tissue-derived models, and molecular characterization of retrospective human materials through biobanks, bioinformatics, and translational pharmacokinetics/pharmacodynamics [3]. The complexity and multifaceted nature of cancer demands a multidisciplinary and collaborative approach that brings together experts from a broader network of stakeholders, including researchers, clinicians, research centers, universities, government agencies, pharmaceutical partners, patient advocacy groups, and philanthropic organizations, all of whom play a vital role in supporting and advancing cancer research.

Therefore, collaboration is the cornerstone of successful large-scale translational research projects in cancer treatment. By fostering collaboration, these projects gain access to a diverse pool of knowledge, skills, and resources, which are essential for tackling the multifaceted challenges of cancer research [4]. Collaboration enhances the speed and efficiency of research, promotes knowledge exchange, and fosters innovation [5] by joining together advanced technologies, methodologies, extensive infrastructure, expertise, and resources, making it possible to undertake ambitious research endeavors that would be challenging for individual institutions to achieve. Collaborating with international partners allows for the exchange of insights and data from diverse patient populations, which can lead to more robust and generalizable findings.

The significant role of digital platforms in translational research has been crucial in providing support for the storage and integration of big data, an analysis context, and additional information from external sources. They were created as a solution to the growing amount of omics data and use informatics methods to link molecular and clinical data [6]. The benefits of these platforms include the ability to identify biomarkers, develop personalized medicine, and improve patient outcomes [7]. It is understood that such platforms enable collaboration by sharing research data and tools, such as in the case of Cancer Core Europe [8] or REDCap [9]. However, their actual impact on collaboration has not received attention in the literature in a systematic manner.

Effectiveness in translational medicine projects rests on seamless collaboration between individuals, teams, and institutional and organizational stakeholders, ensuring that promising discoveries are efficiently tested, validated, and translated into clinical applications that ultimately save lives. However, collaboration is naturally “difficult to manage, and the likelihood of disappointing outputs is high” [10]. However, managing large collaborations proves challenging, often leading to less than satisfactory outcomes [11]. Therefore, assessing collaborative processes is a key activity to reach sustainable research ecosystems that go beyond the period of funding and transform activities in health solutions in the long term. Several evaluation models can be traced in the literature in three categories: indices, processes, and relatedness. Indices are related to bibliometric analysis [12,13,14], processes focus on the method of evaluation and the dimensions that must be analyzed [15,16,17], and relatedness is explored frequently through social network analysis (SNA) [18,19].

Assessing collaboration implies exploring several factors such as planning, infrastructure, relationships, complexity, boundaries, dialogue, and human behavior. Some evaluation models involving these collaboration principles can be traced in the literature, focused mainly on bibliometric indices [12,13,14], collaboration processes [15,16,20], and social network analysis [18,21]. Despite the efforts of these studies, researchers are cautioning that traditional approaches are difficult to operationalize in real contexts [15]. Nevertheless, more empirical research on actual translational projects is necessary, including the conditions and outcomes of collaborative initiatives [22]. To date, interdisciplinary, collaborative, and partnership research in healthcare projects, including researchers’ approaches and actual effects of such programs, is limited, calling for further research. This study evaluates collaboration in a large translational research project related to phytotherapy-based cancer treatment. It adds value by developing a methodological path for evaluation, incorporating qualitative and quantitative data. The Generation of Therapeutic Alternatives in Cancer from Plants (GAT) ecosystem illustrates the evaluation approach using actual center data. Specifically, this study seeks to answer the following questions: How to evaluate the collaboration performance in a translational research ecosystem? What are the main challenges affecting collaboration performance in a translational research ecosystem?

This paper makes the following contributions: the identification of several models to evaluate collaboration in a translational research ecosystem; the determination of variables and barriers to promoting collaboration in large research projects; a set scope of dimensions and factors to govern and manage collaboration effectively; contextual information derived from a real research ecosystem that can guide further projects in the same or in a different discipline.

This paper is organized into five sections, including this introduction. Section 2 reviews the academic literature on collaboration in translational research projects and the current evaluation models. Section 3 describes the research design and context. Section 4 presents the five-step process followed to evaluate collaboration in the GAT ecosystem, including methodological considerations and the corresponding findings and discussions of each phase. Finally, Section 5 concludes the paper.

2. Research Background

2.1. Features, Topics, and Barriers in Scientific Collaboration

Collaboration is defined as “a joint effort towards a common goal” [23]. Collaboration drives complexity in organizational problems and is the key to innovation at the intra- and inter-organizational levels [24,25,26]. In collaborative settings, entities share information, resources, and responsibilities to jointly plan, implement, and evaluate a program of activities to achieve a common goal, thereby jointly generating value [27]. Collaboration leads to enhanced capabilities by sharing engagement, trust, time, effort, and dedication.

Exploring scientific collaboration in large groups implies exploring human, social, and organizational factors. A clinical and translational research project often involves the interaction and input of several stakeholders, such as “researchers, clinicians, pharmacists, statisticians, information technology staff, educators, institutional review board members, administrators, and others” [17].

Inter-organizational collaboration produces some benefits for participating organizations, including access to complementary assets, knowledge exchange, the creation of new knowledge, and the sharing of costs [28]. Collaboration can create value [29] and it also increases productivity [30]. Accordingly, collaborative research incorporates social interactions and a range of potential roles for those involved throughout the research process. Previous studies have outlined means of collaboration in the form of “linkage mechanisms” between researchers and users, such as the involvement of intermediaries (boundary spanners), formal and informal communication with users during the research, the participation of users in data collection, and the provision of interim feedback [22].

Interdisciplinary collaboration has gained importance in the last ten years for developing therapeutic discoveries more quickly than traditional methods and tackling more complicated biological challenges in translational projects. By analyzing 19.9 million papers and 2.1 million patents over 5 decades, Wuchty and colleagues found that “teams increasingly dominate solo authors in the production of knowledge” [31]. Building collaboration in biomedical research at individual and institutional levels leads to improved information sharing between researchers, increased publishing productivity, and new research resources and projects [32].

There are some important reasons for scientists’ collaboration in translational sciences. From a scientific perspective, motivational factors for collaboration are research data availability, co-authoring, synergy of the research, visibility of the research results, the possibility of solving complex research problems, and the legitimacy of one idea or solution. Regarding the financial dimension, research funding and financial incentives are the lead motivators. The social dimension is based on resource availability, such as data, equipment, materials, or technology [30,33,34].

Furthermore, research problems drawing scientists’ attention have changed because they are now more ill-defined, technically complex, and interdisciplinary, requiring highly specialized knowledge of a variety of disciplines [35]. For instance, a university research center can be an expert in developing molecules to reduce the effects of chemotherapy on the nervous system, and a pharmaceutical company can have the knowledge and technology to synthesize the molecules into a commercial product. Therefore, collaboration is now actively encouraged by universities and research institutions [30]. Industry–university collaborations conducting interdisciplinary research are required to solve social and human problems [36] such as COVID-19 [37]. Solutions derived from n-helix initiatives are often accepted as “close prototypes of complex evolving networks” [38]. Therefore, translational research supported by proper public policy can promote technology transfer mechanisms [39], or organizational initiatives such as new research institutes, research centers, and technology centers [15,40]. A comprehensive analysis of the literature about academy–industry–government relationships reveals recurring themes concerning the attributes of collaboration:

• Planning and careful design are relevant in achieving the goals of the collaborative initiatives [41,42,43].
• Collaboration requires a knowledge infrastructure for learning and sharing information [44,45,46].
• Structure and relational elements determine social interaction and the quality of relationships [41,43,44,47].
• Differences between individual and group interests often increase complexity in collaboration [42,48,49].
• Collaboration embraces fuzzy boundaries at intra- and inter-organizational levels [16,50,51].
• Dialogue, reflection, questioning, and clarification inform collaboration performance [52,53].
• Collaboration is grounded in human behaviors within a workgroup setting [47,54].

Collaboration at the institutional level in translational research implies challenges related to differences in governance structures, funding mechanisms, protocols regarding human subjects’ protection, data integrity, authorship criteria, intellectual property laws, conflict resolution, institutional leadership, and support [55]. Moreover, differences in languages and time zones can present practical challenges to communication [56]. In addition, geographic distance and disciplinary disparities can also affect the success of partnerships [57].

Experience in collaboration networks has reduced the barriers of distance or interdisciplinarity [58]. In translational research, researchers prefer to collaborate with researchers with shared experiences and prefer those with whom they have interacted in previous projects [59]. However, collaboration between individuals and teams can also suffer in the absence of supportive and enabling group and institutional leadership [33].

Based on the findings of this literature review, it is apparent that collaboration has been examined from various perspectives, each contributing to our understanding of the concept. However, there is limited research evidence available on an integrated model of collaboration that clearly delineates the factors that precede collaboration, the processes involved, and the potential outcomes that can inform general business practices.

2.2. Evaluating Scientific Collaboration

While organizational collaboration has increased in many fields, providing a means for leveraging fragmented systems and improving efficiency and innovation, it usually does not spontaneously emerge or self-sustain; in fact, most corporate alliances fail [16]. Research collaboration has been recognized as a source of increased knowledge generation, reduced redundancy, and resource savings, as well as a natural consequence of specialization, which requires multiple researchers to address complex research problems [12,30]. Therefore, evaluating collaboration is the keystone for innovation and performance in large research projects.

Certain assessment frameworks that incorporate these principles of collaboration have been identified in existing research. These frameworks primarily concentrate on metrics related to bibliometrics [12,13,14], processes of collaboration [15,16,17], and the analysis of social networks [18,19]. Table 1 presents a summary of the models.

A systematic review found that in evaluating the outcome of biomedical research, collaboration indexes used (and the number of articles that used it) were the number of co-authored publications (4), the number of articles with international collaboration (2), the proportion of long-distance collaborative publications (1), partnership ability index (PHI-index), and dependence degree (d-index) [60]. Bibliometric and networking measurements are often employed to evaluate collaboration. Co-authorship (two or more authors for the same product), co-partners (two or more participants in the same project), and co-cited publications (two or more citing documents to the same document) are some metrics [12]. In evaluating research collaboration, one needs to consider output (co-authorship) as well as the collaboration network itself [12]. In evaluating research projects, although publications are important, they come too late in order to improve deficient collaboration [61]. While using research network links and link weights provides an account of the quantity of collaboration, it does not provide a sense of its quality [12].

Evaluating research collaboration implies the evaluation of the extent of collaboration and quality of communication, the performance of projects and infrastructure, data quality, scientific productivity, and the impact on member organizations [61]. Different processes are required to evaluate the collaboration, such as characterization of collaboration, identification of workgroups, identification of collaboration patterns, monitoring development, evaluating levels of integration, and comparing and analyzing the evolution of collaboration, among others [16,17].

Evaluating collaboration via social network analysis (SNA) is pivotal for understanding interaction dynamics. SNA assesses relationships, revealing key players and information flow [18]. Studies presented in Table 1 remark that measurement variables include centrality metrics (degree, betweenness), indicating influential nodes, and clustering coefficients denoting subgroup cohesion. Structural holes unveil cross-group knowledge opportunities. Evaluating tie strength and connection diversity enhances network robustness insight. SNA not only quantifies collaboration but also uncovers patterns to optimize efforts effectively [19,21].

Table 1. Evaluation models for research collaboration.

Category	Author	Description
Indices	[12]	Evaluating scholars based on their academic collaboration activities. Three researcher and community collaboration indices are proposed: - Co-author collaboration value (CCV): indicates how productive the collaboration between two authors is. - Researcher collaboration index (RC-Index): number of collaborations with qualified co-authors. - Community collaboration index (CC-Index): measures the collaboration activities of a research community.
	[13]	Quantifying the degree of research collaboration. Classic collaborative measures all give accounts of research collaboration: - Collaborative index (CI): average number of authors per paper. - Degree of collaboration (DC): proportion of multiple-authored papers. - Collaborative coefficient (CC): a value between 0 and 1, where 0 corresponds to single-authored papers and 1 to an infinite number of authors. - Revised collaborative coefficient (RCC): adjusts RCC so that 1 can be achieved in maximal collaboration papers.
	[14]	Bibliometric evaluation in translational science. By using bibliometric analysis (number of publications, average number of citations per publication, percentage of publications in the top 10% per citation, comparative citation ratio) the proposal explores the research productivity and influence of the funding actor in collaboration.
Process	[16]	Collaboration Evaluation and Improvement Framework—CEIF. The CEIF involves five phases to evaluate organizational collaboration: (1) the characterization of collaboration; (2) the identification of workgroups; (3) monitoring development; evaluating (4) levels of integration and (5) cycles of inquiry.
	[20]	Analytic model for SCTC research network. The starting point rests on encouraging the formation of new connections between researchers, then connective activities are deployed, and network-level metrics are utilized to measure connections; finally, collaboration outcomes are measured via metrics and performance analysis.
	[62]	Levels of collaboration survey. The survey is designed for those who work for one of the organizations or programs that are partners in the initiative. The model is based on five levels of collaboration: networking, cooperation, coordination, coalition, and collaboration.
	[17]	A relational coordination approach from an organizational perspective. The model is based on some factors such as relational coordination (RC), community engagement (CE), comparative effectiveness research (CER), clinical and translational research (CTR), and relational coordination research collaborative (RCRC).
	[15]	Collaboration performance evaluation in research centers. This research provides a collaboration measurement system for research centers and a decision model to evaluate performance in projects involving government, industry, and academic institutions.
SNA	[18]	Identifying emerging research collaborations and networks. The model affords useful insights for evaluation using SNA to assess networks at several levels of the organization, and link data to assess the evolution of these networks.
	[21]	Visualizing and evaluating the growth of multi-institutional collaboration. It presents a collaboration analysis pipeline based on co-authorship relationship analysis. Results can be used to render and analyze large-scale institutional collaboration.
	[19]	Mapping cross-disciplinary collaboration. It presents a variety of ways of mapping and evaluating the growth of cross-disciplinary partnerships over time. SNA is used to examine the impact of funding on collaboration patterns.

Table 1. Evaluation models for research collaboration.

Category	Author	Description
Indices	[12]	Evaluating scholars based on their academic collaboration activities. Three researcher and community collaboration indices are proposed: - Co-author collaboration value (CCV): indicates how productive the collaboration between two authors is. - Researcher collaboration index (RC-Index): number of collaborations with qualified co-authors. - Community collaboration index (CC-Index): measures the collaboration activities of a research community.
	[13]	Quantifying the degree of research collaboration. Classic collaborative measures all give accounts of research collaboration: - Collaborative index (CI): average number of authors per paper. - Degree of collaboration (DC): proportion of multiple-authored papers. - Collaborative coefficient (CC): a value between 0 and 1, where 0 corresponds to single-authored papers and 1 to an infinite number of authors. - Revised collaborative coefficient (RCC): adjusts RCC so that 1 can be achieved in maximal collaboration papers.
	[14]	Bibliometric evaluation in translational science. By using bibliometric analysis (number of publications, average number of citations per publication, percentage of publications in the top 10% per citation, comparative citation ratio) the proposal explores the research productivity and influence of the funding actor in collaboration.
Process	[16]	Collaboration Evaluation and Improvement Framework—CEIF. The CEIF involves five phases to evaluate organizational collaboration: (1) the characterization of collaboration; (2) the identification of workgroups; (3) monitoring development; evaluating (4) levels of integration and (5) cycles of inquiry.
	[20]	Analytic model for SCTC research network. The starting point rests on encouraging the formation of new connections between researchers, then connective activities are deployed, and network-level metrics are utilized to measure connections; finally, collaboration outcomes are measured via metrics and performance analysis.
	[62]	Levels of collaboration survey. The survey is designed for those who work for one of the organizations or programs that are partners in the initiative. The model is based on five levels of collaboration: networking, cooperation, coordination, coalition, and collaboration.
	[17]	A relational coordination approach from an organizational perspective. The model is based on some factors such as relational coordination (RC), community engagement (CE), comparative effectiveness research (CER), clinical and translational research (CTR), and relational coordination research collaborative (RCRC).
	[15]	Collaboration performance evaluation in research centers. This research provides a collaboration measurement system for research centers and a decision model to evaluate performance in projects involving government, industry, and academic institutions.
SNA	[18]	Identifying emerging research collaborations and networks. The model affords useful insights for evaluation using SNA to assess networks at several levels of the organization, and link data to assess the evolution of these networks.
	[21]	Visualizing and evaluating the growth of multi-institutional collaboration. It presents a collaboration analysis pipeline based on co-authorship relationship analysis. Results can be used to render and analyze large-scale institutional collaboration.
	[19]	Mapping cross-disciplinary collaboration. It presents a variety of ways of mapping and evaluating the growth of cross-disciplinary partnerships over time. SNA is used to examine the impact of funding on collaboration patterns.

3. Research Design

3.1. Research Purpose and Design

The purpose of this research is to evaluate collaboration in the GAT ecosystem through the CEIF model [16]. Woodland and Hutton’s framework was chosen to evaluate collaboration in the GAT ecosystem because it offers a holistic and integral perspective that allows the evaluation of collaborative actions, processes, and activities within scientific research ecosystems, which are characterized by the participation of networks of interdisciplinary actors, the development and evolution of strategic alliances and communities of practice among stakeholders, diverse levels of integration among participants, and different dynamics of coordination and collaboration. These settings are the main characteristics of the GAT ecosystem as one of the first high-scale research programs in Colombia.

In accordance with the research purpose and the CEIF specifications, a mixed research design was applied. Interviews, surveys, and direct observation were used as data collection techniques. Each CEIF phase involved a particular configuration of participants, methodological strategies, and instruments, which are mentioned in the next section. Qualitative data were analyzed through an interpretative approach, and qualitative data were analyzed via descriptive analysis. Some of the CEIF phases require data gathering only from principal investigators (PIs) and others from all the researchers involved. Following the indications of the frameworks, some surveys and interviews were carried out once, and others at the start and end of the program.

3.2. Research Setting: The GAT Ecosystem

The GAT (Generation of Alternative Therapeutics) program is a research collaboration ecosystem that seeks to generate phytomedicines for cancer treatment through translational research processes. The alliance behind GAT is composed of more than one hundred researchers and collaborators from eight Colombian higher education institutions, six international higher education institutions, two Colombian organizations from the productive sector, two international research centers, and six non-profit social institutions. The program is part of the “Colombia Científica” initiative, channeling up to USD 160 million of World Bank funding through ICETEX (Colombia’s public higher education financing organization) and with the participation of Colombia’s Ministries of Education; Science and Technology; and Commerce, Industry, and Tourism.

The program brings together different disciplines and phases along the translational research chain. An ethnobotanical grounding is required to identify the plants with traditional or popular uses, coupled with a study of patients and doctors to determine not just which plants they use but also their attitudes and barriers surrounding phytomedicine for cancer treatment. A biological prospecting stage studies the soil and the plants from which the extracts are obtained; this must be done in conjunction with local farmers and by promoting good agricultural practices. A chemical analysis stage characterizes the extracts of dozens of different plants with potential as well as their polymolecular effects on the tumor microenvironment. As with all drug development, any promising experiments are taken through a series of studies in vitro, in vivo, and all required stages of clinical trials in human patients. Different researchers, in collaboration with partners from the pharmaceutical industry, must then develop effective drug delivery systems with scientific rigor, good manufacturing practices, and in a scalable manner that is amenable to commercialization. All prior steps must be supported by intellectual property, market, and regulatory procedures, including permits for the exploitation of bioeconomic resources, clinical trials, patents, secrets, and local and international food and drug safety approvals.

The five-year program (extended for another year due to the COVID-19 pandemic), is a case of open collaborative science bringing together local and international academia with the government, industry, and civil society. As such, it should account for the development of such collaboration in terms of products, such as publications, PhDs, patents, new products, and services, as well as strengthened capabilities for continued research and development. Reporting performance measures must be done periodically and shared in different scenarios for effective governance and control: at the program level (Technical Committee, IP Committee, International Congress Committee), specifically for the lead university (in charge of receiving, distributing, and administrating all the funding), every two months with the local government actors, and every six months with the World Bank. At the same time, results should be shared with all partners, affected communities, and other interested actors from civil society. Crucially, reporting of performance measures serves for alignment, communication, and administration for the management team and for every principal investigator from each of the ten associated projects.

3.3. Data Collection and Analysis

The data utilized to construct the evaluate collaboration in the GAT ecosystem originated from three primary sources: insights gained from direct observation, interviews with researchers, and surveys. Data collection encompassed document analysis, interviews, and direct observation in the field, guaranteeing objectivity and the quality of findings through the triangulation of multiple data sources (Eisenhardt, 1989). The collected data pertained to each research project undertaken by GAT and also incorporated insights from different stakeholders.

To ensure traceability, the findings were meticulously recorded and stored in a database (Miles and Huberman, 1994), and a structured coding process (Miles and Huberman, 1994) was implemented for the corpus. The analysis of the data employed pattern-matching logic (Yin, 2009). Descriptive analysis was utilized to examine qualitative data collected via surveys.

The data obtained from the surveys was also stored in the database. By consolidating the results from all research instruments, a consolidated analysis of collaboration in the GAT ecosystem was developed. Additionally, data were collected from researchers, including individual notes, surveys, interviews, and reports. During the evaluation of the collaboration, further data were gathered through additional conversations with stakeholders, in order to clarify some points of discussion. To ensure the reproducibility and replicability of this study, all research data were documented and saved in a database (Data Availability Statement).

4. Results and Discussion

4.1. Phase 1—Operationalize Collaboration

According to Woodland and Hutton, “the sine qua non of collaboration is shared purpose” [16]. In this study, we operationalize collaboration by exploring awareness (who knows whom), access (who has what), and engagement (who is able to) [63], which are fundamentals of a knowledge management strategy. In doing so, we use a technology-based strategy by deploying a knowledge management system (KMS) in order to manage the corpus of data-information-knowledge generated in each project. For the purposes of this study, the program adopted the Open Science Framework (OSF) [64], as can be seen in Figure 1 and explored directly (https://osf.io/xrgp2/ accessed on 15 may 2023). All progress reports, student reports, lab reports, and final project reports are stored in the shared project spaces of the OSF. The KMS affords key features to support collaboration, such as editability, modularity, improvisation, tracking, openness, traceability, and uncovering, which are powerful for leveraging collaboration in large groups [65,66]. All the material generated in the GAT projects is available on the platform to guarantee information access and enable assessment and monitoring.

Awareness and engagement are enabled by visualizing the main characteristics of the GAT ecosystem (see https://kumu.io/nestornova/mapa-de-capacidades-gat accessed on 30 may 2023). In doing so, we utilized KUMU [67], which affords meta-knowledge for discovering and sharing knowledge across individual, group, and project ecosystems. Knowledge access is done through an online network visualization displaying relevant information about the configuration of GAT, such as project information, research, and ecosystem capabilities, program infrastructure, organization structure, research phases, and information about study subjects. Figure 2 shows the visualization map.

4.2. Phase 2—Identify and Map Communities of Practice

After identifying the initial configuration of the research ecosystem, we determined the state of the collaboration between GAT participants. We performed a social network analysis (SNA) grounded in [68] by using dynamic data collected through an online questionnaire applied at the beginning of the program (Figure 3a) and after three years (Figure 3b). Items to score collaboration were networking, cooperation, coordination, coalition, and collaboration [27]. We also asked participants to score the frequency of their interactions, the value of the information shared with the research ecosystem, and the coordination mechanisms for sharing. The results in Figure 3 show the strengthening of the network both in terms of ties and structure using standard SNA metrics. In general, density and centrality degrees, as well as diameter, average path length, and reciprocity increased, as shown in Figure 3b. The resulting network exhibits a scale-free topology, which is in line with prior studies in translational research [59,69]. This means that less than 20% of researchers are deeply connected in terms of interdependencies and information exchange and can be considered network hubs, whereas the majority of researchers have limited connections.

4.3. Phase 3—Monitor Stages of Development

In this phase, we went through the exploration of the four stages of development proposed in [16]: the assemble/formation stage, storm/ordering stage, norm/perform, and transform/adjourn. The goal of this phase is to evaluate the movement and evolution of collaboration in GAT, over time, which in turn serves as input for institutional strengthening decisions promoting collaboration. We conducted several interviews with the PIs and researchers of each of the ten projects composing the GAT ecosystem, which were identified through the inventory and mapping process in Phase 2. Interviews were used to identify and manage issues about collaboration quality that require special attention, correction, and improvement. From each set of suggested questions for each stage in the CEIF, we selected the ones that met the GAT ecosystem characteristics. Interviews were conducted at two moments: the assemble/formation stage and norm/perform were explored at the beginning of the program, whereas norm/perform and transform/adjourn were delivered in the third year. For each application, we designed an interview protocol in order to control and monitor the development of the sessions.

Success in this assemble/formation stage of collaboration frequently depends on how clearly everyone understands the aim, structures, tactics, leadership, and important duties.

Table 2. Assemble/formation stage at GAT.

Variable	Insight	Analysis
Linkage to project	Trust in collaboration	In those groups in which there was experience in previous projects, collaboration works better than in those in which such experience is nonexistent.
Member recruitment	Philosophical alignment	Science and collaboration rest on the onto-epistemological alignment between researchers around a research question. Alignment is evident when developing joint research previously.
	Structure and dynamic organization	Relationships between work methodology, hierarchy, bureaucracy, and centralization all impact effectiveness in collaboration. Staff turnover partially affected the development of the project.
Common understanding	Engagement	Difference between technical capacity and research motivation. Commitment to the program depends on the academic level of the student/contractor.
	Leadership	The PI role requires scientific and managerial competencies that can impact the cohesion of the project team.
	Sharing	Relationship between knowledge sharing and effective collaboration.
	Uncertainty	The degree of uncertainty in the variation of the project depends on the results to be obtained.
Workflow	Dynamics	The workflow of the project means that not everyone enters at zero time. Common understanding is critical to proactive work. There is a differential speed between on-site teams and those located in other cities.
	Task division and alignment	There is a division of tasks, each participant knows what each one has to do. Clarity in assignments from the beginning is key and nothing should be an imposition.
	Respectful	There is respect for the different expertise in the project research team and the clinicians.

presents a summary of insights and analysis about the identified variables in the assemble/formation stage.

After the alliance has been formed, a crucial phase of development follows. The discussion of the GAT ecosystem’s common goals frequently involves factors related to awareness, resource scarcity, territorial ambition, domain knowledge, and individual readiness to assume duties.

Table 3. Storm/ordering stage at GAT.

Variable	Insight	Analysis
Purpose	Conscious	All PIs are conscious of the purpose of their project and the program in general.
Goals	Indicators	Knowledge dissemination about the program ecosystem is a key factor in evaluating collaboration success.
Outcomes	Resource flow	Process management is needed to guarantee resource flow among projects.
Norms	Sharing	Recurrent meetings, doctoral seminars, and conferences allow us to keep up-to-date knowledge about the program.
Governance	Monitoring	There are mandatory and contractual guidelines that must be followed to control the program’s execution. Meetings are the scenario to monitor development. Publications are the way to evaluate program outcomes. Oversight allows to monitor technical performance.
Decision making	Consensus	Decisions are consensual because each university has autonomy, but feedback is used to decide. It is important to align the individual interests with the program ones. Researchers’ actions rest on previous meetings aimed at reaching an agreement.
Information dissemination	Localization	Interested in knowing how the project is progressing in their own locality but not so much about the program performance in general.
	Interdependencies	The level of information flow is higher in projects that have greater interdependencies between them at the scientific level.
	Systems	The program information, especially reports, is stored in the OSF in order to centralize critical details to send to funders.

presents a summary of insights and analysis about the identified variables in the storm/ordering stage.

Once the alliance activities have been ordered, the next step is to normalize them looking to increase the research program’s performance. At this time, the technical and administrative systems of the GAT ecosystem have been implemented, and necessary and regular tasks and activities are executed to accomplish the alliance’s goals.

Table 4. Norm/performing stage at GAT.

Variable	Insight	Analysis
Operations	Administration	Management tasks take more time than scientific processes, increasing complexity in the overall ecosystem operation. Proper support of administrative tasks is key to reaching the goal’s project.
	Coordination	Coordination is the keystone to aligning diverse institutions, each one with its own interests.
	Leadership	Even though each project has its own PI, the scientific manager can make decisions that sometimes can be against the Principal Investigators.
	Sharing	Due to the effects of COVID-19’ on the global economy, new capabilities to share resources between laboratories were developed.
	Standardization	Some experiments can be executed nowadays in a standard way, with procedures and times controlled. This makes it easier to sell services.
	Infrastructure	Some collaborative relationships are based on physical spaces such as specialized laboratories, but these relations can be temporal due to the nature of independence between institutions.
Knowledge transfer	Stakeholder turnover	Job turnover considerably affects project performance, especially when one expert or institution leaves.
Networks	Networking	During the program assemblage, each project worked as a self-contained project, but now, there is a network of interdependencies and connections between them allowing to exchange of information and aligning processes and outputs.
	Alignment	The development phases of each project must be respected but require articulation and synchronization for their entry into operation. Alignment does not fully imply an assemblage between the whole projects, whereas it implies the proper collaboration among them at the necessary level.
Collaboration	Purpose	Collaboration can be based on process capabilities, but also in relation to a more particular object of study, such as a specific plant extract.
	Maturity	The level of maturity in the research groups composing the ecosystems determines the effectiveness of collaboration and program performance.
	Planning	Evaluation collaboration also implies assessing planning tasks in order to modify them to reduce risk when developing research.
Technology	Equipment	In some cases, alignment between team rest on technology availability in the laboratory, for instance, special equipment can be utilized by three programs.
	Information systems	The data platforms allow us to perform some experiments and analyses in a more efficient way. The KMS enables information and knowledge sharing among projects and the program governance activities.
Macroeconomy	Money	Currency volatility between countries can affect collaboration leading to the development of activities locally to reduce costs.

presents a summary of insights and analysis about the identified variables in the norm/performing stage.

During the transformation stage, alliances and teams may encounter planned and unexpected events. In this phase, assessing collaboration can help determine whether to improve, restructure, or dissolve it. This leads to three potential outcomes: formal termination, continuation without changes, or modifications to its functioning.

Table 5. Transform/adjourn stage at GAT.

Variable	Insight	Analysis
Vision	Outcomes	Novel results can inform new research project proposals in or out of the ecosystem.
Networks	Evolution	Collaborative networks to perform the project can derive new alliances to conduct additional investigations.
	Alignment	All the Principal Investigators should be fully connected and articulated to get the expected performance.
Collaboration	Capabilities	Collaborative work allows for reconfiguring the scientific ecosystem to research new and novel topics that emerged from the program development, but also contingencies related to public health such as COVID-19.
Trust	Trust development	The execution of the tasks over time can determine the need to replace a researcher due to engagement.
Institutionality	Engagement	Some institutions are invited to make part of the ecosystem but do not contribute as expected.
	Mentorship	Less experienced institutions require mentorship at scientific and administrative levels to produce the expected outcomes effectively.
	Sustainability	New generations of researchers should be trained in administrative and scientific capabilities to lead large research projects taking into account the lessons learned and best practices derived from the GAT ecosystem.
	Funding	Evaluating collaboration also implies assessing support, opportunity, administration, and management from the funding actors and intermediaries.

presents a summary of insights and analysis about the identified variables in the transform/adjourn stage.

4.4. Phase 4—Assess Levels of Integration

In the CEIF, Woodland and Hutton mention that collaboration relies on integrations between and within organizations, however, the level of integration should meet the purpose and goals of the alliance, because “more integration is not necessarily better” [16]. To explore integration in the GAT ecosystem, we customized and applied the Levels of Organizational Integration Rubric (LOIR) [16] to the PIs and researchers of each project in the GAT ecosystem. The LOIR describes five levels of organizational integration (from zero/none to four) and the purposes, strategies/tasks, leadership/decision making, and communication characteristics that tend to be present at each level of integration [16]. We asked participants to answer questions about the actual and desired level of integration of each project regarding the GAT ecosystem in each variable of LOIR. We use the levels of collaboration proposed in [27] to categorize the results in terms of networking, coordination, cooperation, and collaboration. We also asked open questions about further and particular actions to increase the level of integration.

Figure 4 shows the quantitative results of the survey. In terms of purpose, whereas the participants consider that they work together to ensure that complementary tasks are carried out (coordination), they expect to form a unified single structure (collaboration). The difference between the current and desired results is −1.56. Regarding strategies and tasks, participants consider that the program develops relationships based on consultancies and there are few complementary goals and tasks (coordination); however, they expect a formal structure to support strategies and tasks enacting a joint mission (collaboration), with a difference of −1.44. In leadership and decision making, participants consider that the proper level should represent centralized leadership and clear decision-making mechanisms (cooperation), whereas the current level is based on a non-hierarchical decision-making structure and volunteer leadership. The difference for this category was −1.11. Finally, participants consider that the current level of communication is clear but informal with sporadic conflicts, but they expect to have a more formal communication system. The difference for this category was −1.11. In the leadership/decision-making and communication categories, researchers do not expect to have a full level of integration. Consequently, as the authors of [48] highlight, meaningful integration cannot be simplified; instead, it may call for ineffective interactions to highlight synergies between disciplines.

4.5. Phase 5—Assess Cycles of Inquiry

The final phase of the CEIF is about assessing the quality of interprofessional collaboration [16]. Effective and efficient interprofessional collaboration rests on individual and team capabilities to collaborate, even if people are not skilled in the practice of collaboration itself. One of the most important questions that emerges when evaluating collaboration in large research groups is whether relationships are leading to the expected organizational performance. Sometimes leaders assume that researchers know how to collaborate, and all the alliances will work very well. In this phase, we use the mapping strategies in Phase 2 to examine the cycles of inquiry in the GAT ecosystem. In doing so, we applied the Team Collaboration Assessment Rubric (TCAR) [16] to 39 out of 115 researchers in the GAT ecosystem. The evaluation is operationalized by exploring four characteristics, dialogue, decision-making, action, and evaluation, at different quantities of attributes and levels of quality (2, 1, and 0).

Table 6. TCAR results for the GAT ecosystem.

Project	Dialogue	Decision Making	Action	Evaluation	Average Per Project
P1	1.79	1.93	2.00	1.70	1.85
P2	1.64	1.60	1.49	1.44	1.54
P3	1.71	1.67	1.53	1.60	1.63
P4	1.71	1.52	1.47	1.27	1.49
P5	1.78	1.82	1.63	1.69	1.73
P6	1.64	1.57	1.70	1.50	1.60
P7	1.64	1.57	1.50	2.00	1.68
P8	1.62	1.81	1.73	1.73	1.72
P9	1.86	1.79	2.00	1.80	1.86
P10	1.64	1.61	1.55	1.55	1.59
Average per concept	1.70	1.69	1.66	1.63	1.67

presents the results for this phase.

The results show regularity in the scores around 1.65/2.0, where the strongest dimension of collaboration is dialogue (1.70), and evaluation has the lowest score (1.63). There are no significant differences in the results when discriminating between PIs and other researchers, and only slight differences in the dimensions of action and evaluation. The best performance in terms of interprofessional collaboration is in projects 1 and 9, which are related to the ethnobotanic study and clinical trials of P2Et extract, respectively, both directly related to patients. Project 2, the largest and most heterogeneous in the GAT ecosystem, has one of the lowest average scores for the four concepts. In terms of [48], large research groups are more productive than smaller ones, but heterogeneity in disciplines and institutions, together with coordination complexity, can affect performance.

4.6. Summary of Phases Insights

Supporting collaboration can be achieved through technology-based strategies, such as deploying a knowledge management system (KMS) and utilizing online network visualizations, which enable communication and information exchange at the scientific and management levels, facilitating awareness, access, and engagement among researchers. This means that coordination mechanisms are key to promoting collaboration [17]. Promoting adequate networking is also a valuable tool to strengthen the GAT ecosystem over time. “Knowing who” is the most effective way to coordinate interdependencies between activities and information exchange between people, teams, institutions, and research ecosystems. Therefore, even though collaboration can be highly supported by technology, human relations are the core to leverage it [58].

By monitoring the five stages of development in the GAT ecosystem, it is possible to reveal collaboration issues that can affect the ecosystem’s performance. The results suggest that the program aims to move from coordination to collaboration, in a scalable way, emphasizing the need for a unified single structure and formal communication system to support joint missions and decision making. At this point, the GAT ecosystem recognizes the importance of integration of individuals, teams, and institutions conforming to the ecosystem; however, proper levels of integration should meet the specific and particular collaboration requirements of each project, avoiding standardization of levels. Finally, interprofessional collaboration rests on a collaborative culture in which training and support are key variables.

5. Conclusions

This paper reports the evaluation of collaboration in a large translational research program about cancer treatment. We ground our work on the CEIF. Our qualitative and quantitative analyses suggest that collaboration is crucial for innovation in translational projects, enhancing capabilities by sharing knowledge resources, materials, and expertise. Collaboration drives knowledge production and value creation, and it has become increasingly important in addressing the complex challenges of scientific investigation, leading to increased team-based research output. However, successful implementation requires careful planning and knowledge infrastructure.

We identified several challenges affecting collaboration performance, such as governance, management, processes, and operations, leadership, and social and human relationships. By exploring and analyzing these insights, research project leaders and stakeholders can monitor the progress of the ecosystems and consistently make informed decisions on enhancing the performance of the collaboration. Our insights also lead to considering that evaluating collaboration implies not only quantifying the outputs based on traditional metrics such as the number of papers, conferences, and so on. Evaluation attempts to analyze in a holistic way how leaders can better guide the research projects in order to reach cohesion, synergy, and cooperation among stakeholders. Additionally, the process we conducted, and the corresponding results can also guide researchers when designing a new research ecosystem, aiming at sustainability in a current one, or even deciding when it should be adjourned. All of these decisions are valuable information for the research ecosystem performance evaluation.

This study demonstrates that technology-based strategies, networking, a systematic evaluation of collaboration stages, integration, and interprofessional collaboration are crucial in promoting effective collaboration in a translational research program. These findings provide valuable insights for managing and improving collaborative efforts in large research groups. The results allow us to anticipate collaboration issues with actionable and opportune strategies that can enhance the planning process, ecosystem performance, sustainability, research outcomes, profit, and the program’s overall success.

The novelty of this study rests on evaluating the collaboration conditions in translational research projects leading to sustainability, as posited in [62], of the services developed during the execution, as well as identifying reproducibility conditions of this initiative in other disciplines and sectors different from cancer research.

The findings of this paper are specific to translational research programs and cannot be directly applied to other sectors. The generalizability of these findings will require investigation in other private or public sectors and with a different unit of analysis. Reproducibility studies in other research ecosystems currently being executed in Colombia in sectors such as sustainable energy, sustainable agricultural crops, productive and social inclusion, and others, could potentially lead to the generalization of collaboration factors identified in this study. These further studies could also promote adjustments in public policy, enabling scientific collaboration.

Even though the GAT ecosystem involves international actors, we focused on local actors, because cultural differences in collaboration research can generate bias: future studies could explore the multinational aspects related to collaboration in translational research. During the four stages of development in Phase 3, we identified several variables impacting collaboration performance that merit further analysis in order to determine the relative impact of each one. Incorporating this broader perspective in future studies would capture the diverse dynamics of collaboration in large research programs.