Evaluating Collaboration in a Translational Research Ecosystem

: A core challenge of a multidisciplinary and multi-organizational translational research program is to set up and promote collaboration between researchers, labs, and organizations. Although the literature has studied and provided guidelines for collaboration, little has been written on how to evaluate it in large research projects and in a practical way. This study aims to identify dimensions and barriers to evaluating and leveraging collaboration in a large translational research ecosystem re-lated to developing phytotherapy-based cancer treatments. By applying the Collaboration Evaluation and Improvement Framework (CEIF), our paper adds value by developing a methodological design for evaluation, incorporating mixed data in a real research ecosystem. Empirical ﬁndings provide support for applying the assessment approach and show that a research project’s sustainability depends on several collaboration factors and barriers at the socio-technical, management, operational, and institutional levels. Research results provide valuable insights for managing and improving collaborative efforts in large research groups, by anticipating collaboration issues with actionable and opportune strategies that can enhance the planning process


Introduction
Translational research bridges the gap between scientific discoveries and practical healthcare, comprising four phases: T1 (basic research), T2 (preclinical studies), T3 (clinical trials), and T4 (implementation and dissemination).The goal of translational research is to bridge the gap between basic science and clinical practice, with the ultimate aim of improving patient outcomes [1].This has been particularly relevant in cancer research because of recent advances in our understanding of the molecular characterization of tumors and the tumor microenvironment, enabling the development of novel treatments [2].According to Dolgos et al. (2016), by integrating basic science research with clinical development, translational medicine has helped to improve target understanding, patient subpopulation selection, and organizational learning.It has also led to advances in areas such as disease model validation, human cell-or tissue-derived models, and molecular characterization of retrospective human materials through biobanks, bioinformatics, and translational pharmacokinetics/pharmacodynamics [3].The complexity and multifaceted nature of cancer demands a multidisciplinary and collaborative approach that brings together experts from a broader network of stakeholders, including researchers, clinicians, research centers, universities, government agencies, pharmaceutical partners, patient advocacy groups, and philanthropic organizations, all of whom play a vital role in supporting and advancing cancer research.This paper is organized into five sections, including this introduction.Section 2 reviews the academic literature on collaboration in translational research projects and the current evaluation models.Section 3 describes the research design and context.Section 4 presents the five-step process followed to evaluate collaboration in the GAT ecosystem, including methodological considerations and the corresponding findings and discussions of each phase.Finally, Section 5 concludes the paper.

Features, Topics, and Barriers in Scientific Collaboration
Collaboration is defined as "a joint effort towards a common goal" [23].Collaboration drives complexity in organizational problems and is the key to innovation at the intraand inter-organizational levels [24][25][26].In collaborative settings, entities share information, resources, and responsibilities to jointly plan, implement, and evaluate a program of activities to achieve a common goal, thereby jointly generating value [27].Collaboration leads to enhanced capabilities by sharing engagement, trust, time, effort, and dedication.
Exploring scientific collaboration in large groups implies exploring human, social, and organizational factors.A clinical and translational research project often involves the interaction and input of several stakeholders, such as "researchers, clinicians, pharmacists, statisticians, information technology staff, educators, institutional review board members, administrators, and others" [17].
Inter-organizational collaboration produces some benefits for participating organizations, including access to complementary assets, knowledge exchange, the creation of new knowledge, and the sharing of costs [28].Collaboration can create value [29] and it also increases productivity [30].Accordingly, collaborative research incorporates social interactions and a range of potential roles for those involved throughout the research process.Previous studies have outlined means of collaboration in the form of "linkage mechanisms" between researchers and users, such as the involvement of intermediaries (boundary spanners), formal and informal communication with users during the research, the participation of users in data collection, and the provision of interim feedback [22].
Interdisciplinary collaboration has gained importance in the last ten years for developing therapeutic discoveries more quickly than traditional methods and tackling more complicated biological challenges in translational projects.By analyzing 19.9 million papers and 2.1 million patents over 5 decades, Wuchty and colleagues found that "teams increasingly dominate solo authors in the production of knowledge" [31].Building collaboration in biomedical research at individual and institutional levels leads to improved information sharing between researchers, increased publishing productivity, and new research resources and projects [32].
There are some important reasons for scientists' collaboration in translational sciences.From a scientific perspective, motivational factors for collaboration are research data availability, co-authoring, synergy of the research, visibility of the research results, the possibility of solving complex research problems, and the legitimacy of one idea or solution.Regarding the financial dimension, research funding and financial incentives are the lead motivators.The social dimension is based on resource availability, such as data, equipment, materials, or technology [30,33,34].
Furthermore, research problems drawing scientists' attention have changed because they are now more ill-defined, technically complex, and interdisciplinary, requiring highly specialized knowledge of a variety of disciplines [35].For instance, a university research center can be an expert in developing molecules to reduce the effects of chemotherapy on the nervous system, and a pharmaceutical company can have the knowledge and technology to synthesize the molecules into a commercial product.Therefore, collaboration is now actively encouraged by universities and research institutions [30].Industry-university collaborations conducting interdisciplinary research are required to solve social and human problems [36] such as COVID-19 [37].Solutions derived from n-helix initiatives are often accepted as "close prototypes of complex evolving networks" [38].Therefore, translational research supported by proper public policy can promote technology transfer mechanisms [39], or organizational initiatives such as new research institutes, research centers, and technology centers [15,40].A comprehensive analysis of the literature about academy-industry-government relationships reveals recurring themes concerning the attributes of collaboration:

•
Planning and careful design are relevant in achieving the goals of the collaborative initiatives [41][42][43].

•
Collaboration is grounded in human behaviors within a workgroup setting [47,54].
Collaboration at the institutional level in translational research implies challenges related to differences in governance structures, funding mechanisms, protocols regarding human subjects' protection, data integrity, authorship criteria, intellectual property laws, conflict resolution, institutional leadership, and support [55].Moreover, differences in languages and time zones can present practical challenges to communication [56].In addition, geographic distance and disciplinary disparities can also affect the success of partnerships [57].
Experience in collaboration networks has reduced the barriers of distance or interdisciplinarity [58].In translational research, researchers prefer to collaborate with researchers with shared experiences and prefer those with whom they have interacted in previous projects [59].However, collaboration between individuals and teams can also suffer in the absence of supportive and enabling group and institutional leadership [33].
Based on the findings of this literature review, it is apparent that collaboration has been examined from various perspectives, each contributing to our understanding of the concept.However, there is limited research evidence available on an integrated model of collaboration that clearly delineates the factors that precede collaboration, the processes involved, and the potential outcomes that can inform general business practices.

Evaluating Scientific Collaboration
While organizational collaboration has increased in many fields, providing a means for leveraging fragmented systems and improving efficiency and innovation, it usually does not spontaneously emerge or self-sustain; in fact, most corporate alliances fail [16].Research collaboration has been recognized as a source of increased knowledge generation, reduced redundancy, and resource savings, as well as a natural consequence of specialization, which requires multiple researchers to address complex research problems [12,30].Therefore, evaluating collaboration is the keystone for innovation and performance in large research projects.
Certain assessment frameworks that incorporate these principles of collaboration have been identified in existing research.These frameworks primarily concentrate on metrics related to bibliometrics [12][13][14], processes of collaboration [15][16][17], and the analysis of social networks [18,19].Table 1 presents a summary of the models.
A systematic review found that in evaluating the outcome of biomedical research, collaboration indexes used (and the number of articles that used it) were the number of co-authored publications (4), the number of articles with international collaboration (2), the proportion of long-distance collaborative publications (1), partnership ability index (PHIindex), and dependence degree (d-index) [60].Bibliometric and networking measurements are often employed to evaluate collaboration.Co-authorship (two or more authors for the same product), co-partners (two or more participants in the same project), and co-cited publications (two or more citing documents to the same document) are some metrics [12].In evaluating research collaboration, one needs to consider output (co-authorship) as well as the collaboration network itself [12].In evaluating research projects, although publications are important, they come too late in order to improve deficient collaboration [61].While using research network links and link weights provides an account of the quantity of collaboration, it does not provide a sense of its quality [12].
Evaluating research collaboration implies the evaluation of the extent of collaboration and quality of communication, the performance of projects and infrastructure, data quality, scientific productivity, and the impact on member organizations [61].Different processes are required to evaluate the collaboration, such as characterization of collaboration, identification of workgroups, identification of collaboration patterns, monitoring development, evaluating levels of integration, and comparing and analyzing the evolution of collaboration, among others [16,17].
Evaluating collaboration via social network analysis (SNA) is pivotal for understanding interaction dynamics.SNA assesses relationships, revealing key players and information flow [18].Studies presented in Table 1 remark that measurement variables include centrality metrics (degree, betweenness), indicating influential nodes, and clustering coefficients denoting subgroup cohesion.Structural holes unveil cross-group knowledge opportunities.Evaluating tie strength and connection diversity enhances network robustness insight.SNA not only quantifies collaboration but also uncovers patterns to optimize efforts effectively [19,21].

Category Author Description
Indices [12] Evaluating scholars based on their academic collaboration activities.Collaborative coefficient (CC): a value between 0 and 1, where 0 corresponds to single-authored papers and 1 to an infinite number of authors.-Revised collaborative coefficient (RCC): adjusts RCC so that 1 can be achieved in maximal collaboration papers.[14] Bibliometric evaluation in translational science.By using bibliometric analysis (number of publications, average number of citations per publication, percentage of publications in the top 10% per citation, comparative citation ratio) the proposal explores the research productivity and influence of the funding actor in collaboration.

Category Author Description
Process [16] Collaboration Evaluation and Improvement Framework-CEIF.
The CEIF involves five phases to evaluate organizational collaboration: (1) the characterization of collaboration; (2) the identification of workgroups; (3) monitoring development; evaluating (4) levels of integration and (5) cycles of inquiry. [20] Analytic model for SCTC research network.
The starting point rests on encouraging the formation of new connections between researchers, then connective activities are deployed, and network-level metrics are utilized to measure connections; finally, collaboration outcomes are measured via metrics and performance analysis. [62] Levels of collaboration survey.The survey is designed for those who work for one of the organizations or programs that are partners in the initiative.The model is based on five levels of collaboration: networking, cooperation, coordination, coalition, and collaboration. [17] A relational coordination approach from an organizational perspective.
The model is based on some factors such as relational coordination (RC), community engagement (CE), comparative effectiveness research (CER), clinical and translational research (CTR), and relational coordination research collaborative (RCRC). [15] Collaboration performance evaluation in research centers.This research provides a collaboration measurement system for research centers and a decision model to evaluate performance in projects involving government, industry, and academic institutions.
SNA [18] Identifying emerging research collaborations and networks.The model affords useful insights for evaluation using SNA to assess networks at several levels of the organization, and link data to assess the evolution of these networks. [21] Visualizing and evaluating the growth of multi-institutional collaboration.
It presents a collaboration analysis pipeline based on co-authorship relationship analysis.Results can be used to render and analyze large-scale institutional collaboration. [19] Mapping cross-disciplinary collaboration.
It presents a variety of ways of mapping and evaluating the growth of cross-disciplinary partnerships over time.SNA is used to examine the impact of funding on collaboration patterns.

Research Purpose and Design
The purpose of this research is to evaluate collaboration in the GAT ecosystem through the CEIF model [16].Woodland and Hutton's framework was chosen to evaluate collaboration in the GAT ecosystem because it offers a holistic and integral perspective that allows the evaluation of collaborative actions, processes, and activities within scientific research ecosystems, which are characterized by the participation of networks of interdisciplinary actors, the development and evolution of strategic alliances and communities of practice among stakeholders, diverse levels of integration among participants, and different dynamics of coordination and collaboration.These settings are the main characteristics of the GAT ecosystem as one of the first high-scale research programs in Colombia.
In accordance with the research purpose and the CEIF specifications, a mixed research design was applied.Interviews, surveys, and direct observation were used as data collection techniques.Each CEIF phase involved a particular configuration of participants, methodological strategies, and instruments, which are mentioned in the next section.Qualitative data were analyzed through an interpretative approach, and qualitative data were analyzed via descriptive analysis.Some of the CEIF phases require data gathering only from principal investigators (PIs) and others from all the researchers involved.Following the indications of the frameworks, some surveys and interviews were carried out once, and others at the start and end of the program.

Research Setting: The GAT Ecosystem
The GAT (Generation of Alternative Therapeutics) program is a research collaboration ecosystem that seeks to generate phytomedicines for cancer treatment through translational research processes.The alliance behind GAT is composed of more than one hundred researchers and collaborators from eight Colombian higher education institutions, six international higher education institutions, two Colombian organizations from the productive sector, two international research centers, and six non-profit social institutions.The program is part of the "Colombia Científica" initiative, channeling up to USD 160 million of World Bank funding through ICETEX (Colombia's public higher education financing organization) and with the participation of Colombia's Ministries of Education; Science and Technology; and Commerce, Industry, and Tourism.
The program brings together different disciplines and phases along the translational research chain.An ethnobotanical grounding is required to identify the plants with traditional or popular uses, coupled with a study of patients and doctors to determine not just which plants they use but also their attitudes and barriers surrounding phytomedicine for cancer treatment.A biological prospecting stage studies the soil and the plants from which the extracts are obtained; this must be done in conjunction with local farmers and by promoting good agricultural practices.A chemical analysis stage characterizes the extracts of dozens of different plants with potential as well as their polymolecular effects on the tumor microenvironment.As with all drug development, any promising experiments are taken through a series of studies in vitro, in vivo, and all required stages of clinical trials in human patients.Different researchers, in collaboration with partners from the pharmaceutical industry, must then develop effective drug delivery systems with scientific rigor, good manufacturing practices, and in a scalable manner that is amenable to commercialization.All prior steps must be supported by intellectual property, market, and regulatory procedures, including permits for the exploitation of bioeconomic resources, clinical trials, patents, secrets, and local and international food and drug safety approvals.
The five-year program (extended for another year due to the COVID-19 pandemic), is a case of open collaborative science bringing together local and international academia with the government, industry, and civil society.As such, it should account for the development of such collaboration in terms of products, such as publications, PhDs, patents, new products, and services, as well as strengthened capabilities for continued research and development.Reporting performance measures must be done periodically and shared in different scenarios for effective governance and control: at the program level (Technical Committee, IP Committee, International Congress Committee), specifically for the lead university (in charge of receiving, distributing, and administrating all the funding), every two months with the local government actors, and every six months with the World Bank.At the same time, results should be shared with all partners, affected communities, and other interested actors from civil society.Crucially, reporting of performance measures serves for alignment, communication, and administration for the management team and for every principal investigator from each of the ten associated projects.

Data Collection and Analysis
The data utilized to construct the evaluate collaboration in the GAT ecosystem originated from three primary sources: insights gained from direct observation, interviews with researchers, and surveys.Data collection encompassed document analysis, interviews, and direct observation in the field, guaranteeing objectivity and the quality of findings through the triangulation of multiple data sources (Eisenhardt, 1989).The collected data pertained to each research project undertaken by GAT and also incorporated insights from different stakeholders.
To ensure traceability, the findings were meticulously recorded and stored in a database (Miles and Huberman, 1994), and a structured coding process (Miles and Huberman, 1994) was implemented for the corpus.The analysis of the data employed pattern-matching logic (Yin, 2009).Descriptive analysis was utilized to examine qualitative data collected via surveys.
The data obtained from the surveys was also stored in the database.By consolidating the results from all research instruments, a consolidated analysis of collaboration in the GAT ecosystem was developed.Additionally, data were collected from researchers, including individual notes, surveys, interviews, and reports.During the evaluation of the collaboration, further data were gathered through additional conversations with stakeholders, in order to clarify some points of discussion.To ensure the reproducibility and replicability of this study, all research data were documented and saved in a database (Data Availability Statement).

Phase 1-Operationalize Collaboration
According to Woodland and Hutton, "the sine qua non of collaboration is shared purpose" [16].In this study, we operationalize collaboration by exploring awareness (who knows whom), access (who has what), and engagement (who is able to) [63], which are fundamentals of a knowledge management strategy.In doing so, we use a technologybased strategy by deploying a knowledge management system (KMS) in order to manage the corpus of data-information-knowledge generated in each project.For the purposes of this study, the program adopted the Open Science Framework (OSF) [64], as can be seen in Figure 1 and explored directly (https://osf.io/xrgp2/accessed on 15 may 2023).All progress reports, student reports, lab reports, and final project reports are stored in the shared project spaces of the OSF.The KMS affords key features to support collaboration, such as editability, modularity, improvisation, tracking, openness, traceability, and uncovering, which are powerful for leveraging collaboration in large groups [65,66].All the material generated in the GAT projects is available on the platform to guarantee information access and enable assessment and monitoring.
base (Miles and Huberman, 1994), and a structured coding process (Miles and Hu 1994) was implemented for the corpus.The analysis of the data employed pattern ing logic (Yin, 2009).Descriptive analysis was utilized to examine qualitative d lected via surveys.
The data obtained from the surveys was also stored in the database.By conso the results from all research instruments, a consolidated analysis of collaboratio GAT ecosystem was developed.Additionally, data were collected from researc cluding individual notes, surveys, interviews, and reports.During the evaluatio collaboration, further data were gathered through additional conversations wi holders, in order to clarify some points of discussion.To ensure the reproducib replicability of this study, all research data were documented and saved in a d (Data Availability Statement).

Phase 1-Operationalize Collaboration
According to Woodland and Hutton, "the sine qua non of collaboration i purpose" [16].In this study, we operationalize collaboration by exploring awaren knows whom), access (who has what), and engagement (who is able to) [63], w fundamentals of a knowledge management strategy.In doing so, we use a tech based strategy by deploying a knowledge management system (KMS) in order to the corpus of data-information-knowledge generated in each project.For the pur this study, the program adopted the Open Science Framework (OSF) [64], as can in Figure 1 and explored directly (https://osf.io/xrgp2/accessed on 15 may 2023).gress reports, student reports, lab reports, and final project reports are stored in th project spaces of the OSF.The KMS affords key features to support collaboration editability, modularity, improvisation, tracking, openness, traceability, and unc which are powerful for leveraging collaboration in large groups [65,66].All the generated in the GAT projects is available on the platform to guarantee informatio and enable assessment and monitoring.Awareness and engagement are enabled by visualizing the main characteristics of the GAT ecosystem (see https://kumu.io/nestornova/mapa-de-capacidades-gataccessed on 30 may 2023).In doing so, we utilized KUMU [67], which affords meta-knowledge for discovering and sharing knowledge across individual, group, and project ecosystems.Knowledge access is done through an online network visualization displaying relevant information about the configuration of GAT, such as project information, research, and ecosystem capabilities, program infrastructure, organization structure, research phases, and information about study subjects.Figure 2 shows the visualization map.

PEER REVIEW 9 of 19
Awareness and engagement are enabled by visualizing the main characteristics of the GAT ecosystem (see https://kumu.io/nestornova/mapa-de-capacidades-gataccessed on 30 may 2023).In doing so, we utilized KUMU [67], which affords meta-knowledge for discovering and sharing knowledge across individual, group, and project ecosystems.Knowledge access is done through an online network visualization displaying relevant information about the configuration of GAT, such as project information, research, and ecosystem capabilities, program infrastructure, organization structure, research phases, and information about study subjects.Figure 2 shows the visualization map.

Phase 2-Identify and Map Communities of Practice
After identifying the initial configuration of the research ecosystem, we determined the state of the collaboration between GAT participants.We performed a social network analysis (SNA) grounded in [68] by using dynamic data collected through an online questionnaire applied at the beginning of the program (Figure 3a) and after three years (Figure 3b).Items to score collaboration were networking, cooperation, coordination, coalition, and collaboration [27].We also asked participants to score the frequency of their interactions, the value of the information shared with the research ecosystem, and the coordination mechanisms for sharing.The results in Figure 3 show the strengthening of the network both in terms of ties and structure using standard SNA metrics.In general, density and centrality degrees, as well as diameter, average path length, and reciprocity increased, as shown in Figure 3b.The resulting network exhibits a scale-free topology, which is in line with prior studies in translational research [59,69].This means that less than 20% of researchers are deeply connected in terms of interdependencies and information exchange and can be considered network hubs, whereas the majority of researchers have limited connections.

Phase 2-Identify and Map Communities of Practice
After identifying the initial configuration of the research ecosystem, we determined the state of the collaboration between GAT participants.We performed a social network analysis (SNA) grounded in [68] by using dynamic data collected through an online questionnaire applied at the beginning of the program (Figure 3a) and after three years (Figure 3b).Items to score collaboration were networking, cooperation, coordination, coalition, and collaboration [27].We also asked participants to score the frequency of their interactions, the value of the information shared with the research ecosystem, and the coordination mechanisms for sharing.The results in Figure 3 show the strengthening of the network both in terms of ties and structure using standard SNA metrics.In general, density and centrality degrees, as well as diameter, average path length, and reciprocity increased, as shown in Figure 3b.The resulting network exhibits a scale-free topology, which is in line with prior studies in translational research [59,69].This means that less than 20% of researchers are deeply connected in terms of interdependencies and information exchange and can be considered network hubs, whereas the majority of researchers have limited connections.

Phase 3-Monitor Stages of Development
In this phase, we went through the exploration of the four stages of development proposed in [16]: the assemble/formation stage, storm/ordering stage, norm/perform, and transform/adjourn.The goal of this phase is to evaluate the movement and evolution of collaboration in GAT, over time, which in turn serves as input for institutional strengthening decisions promoting collaboration.We conducted several interviews with the PIs and researchers of each of the ten projects composing the GAT ecosystem, which were identified through the inventory and mapping process in Phase 2. Interviews were used to identify and manage issues about collaboration quality that require special attention, correction, and improvement.From each set of suggested questions for each stage in the CEIF, we selected the ones that met the GAT ecosystem characteristics.Interviews were conducted at two moments: the assemble/formation stage and norm/perform were explored at the beginning of the program, whereas norm/perform and transform/adjourn were delivered in the third year.For each application, we designed an interview protocol in order to control and monitor the development of the sessions.
Success in this assemble/formation stage of collaboration frequently depends on how clearly everyone understands the aim, structures, tactics, leadership, and important duties.Table 2 presents a summary of insights and analysis about the identified variables in the assemble/formation stage.

Variable
Insight Analysis Linkage to project

Trust in collaboration
In those groups in which there was experience in previous projects, collaboration works better than in those in which such experience is nonexistent.

Philosophical alignment
Science and collaboration rest on the onto-epistemological alignment between researchers around a research question.Alignment is evident when developing joint research previously.

Structure and dynamic organization
Relationships between work methodology, hierarchy, bureaucracy, and centralization all impact effectiveness in collaboration.Staff turnover partially affected the development of the project.

Phase 3-Monitor Stages of Development
In this phase, we went through the exploration of the four stages of development proposed in [16]: the assemble/formation stage, storm/ordering stage, norm/perform, and transform/adjourn.The goal of this phase is to evaluate the movement and evolution of collaboration in GAT, over time, which in turn serves as input for institutional strengthening decisions promoting collaboration.We conducted several interviews with the PIs and researchers of each of the ten projects composing the GAT ecosystem, which were identified through the inventory and mapping process in Phase 2. Interviews were used to identify and manage issues about collaboration quality that require special attention, correction, and improvement.From each set of suggested questions for each stage in the CEIF, we selected the ones that met the GAT ecosystem characteristics.Interviews were conducted at two moments: the assemble/formation stage and norm/perform were explored at the beginning of the program, whereas norm/perform and transform/adjourn were delivered in the third year.For each application, we designed an interview protocol in order to control and monitor the development of the sessions.
Success in this assemble/formation stage of collaboration frequently depends on how clearly everyone understands the aim, structures, tactics, leadership, and important duties.Table 2 presents a summary of insights and analysis about the identified variables in the assemble/formation stage.
After the alliance has been formed, a crucial phase of development follows.The discussion of the GAT ecosystem's common goals frequently involves factors related to awareness, resource scarcity, territorial ambition, domain knowledge, and individual readiness to assume duties.Table 3 presents a summary of insights and analysis about the identified variables in the storm/ordering stage.

Linkage to project Trust in collaboration
In those groups in which there was experience in previous projects, collaboration works better than in those in which such experience is nonexistent.

Philosophical alignment
Science and collaboration rest on the onto-epistemological alignment between researchers around a research question.Alignment is evident when developing joint research previously.

Structure and dynamic organization
Relationships between work methodology, hierarchy, bureaucracy, and centralization all impact effectiveness in collaboration.Staff turnover partially affected the development of the project.

Engagement
Difference between technical capacity and research motivation.Commitment to the program depends on the academic level of the student/contractor.

Leadership
The PI role requires scientific and managerial competencies that can impact the cohesion of the project team.

Sharing
Relationship between knowledge sharing and effective collaboration.

Uncertainty
The degree of uncertainty in the variation of the project depends on the results to be obtained.

Dynamics
The workflow of the project means that not everyone enters at zero time.
Common understanding is critical to proactive work.There is a differential speed between on-site teams and those located in other cities.

Task division and alignment
There is a division of tasks, each participant knows what each one has to do.Clarity in assignments from the beginning is key and nothing should be an imposition.

Respectful
There is respect for the different expertise in the project research team and the clinicians.

Governance Monitoring
There are mandatory and contractual guidelines that must be followed to control the program's execution.Meetings are the scenario to monitor development.Publications are the way to evaluate program outcomes.Oversight allows to monitor technical performance.

Decision making Consensus
Decisions are consensual because each university has autonomy, but feedback is used to decide.It is important to align the individual interests with the program ones.
Researchers' actions rest on previous meetings aimed at reaching an agreement.

Localization
Interested in knowing how the project is progressing in their own locality but not so much about the program performance in general.

Interdependencies
The level of information flow is higher in projects that have greater interdependencies between them at the scientific level.

Systems
The program information, especially reports, is stored in the OSF in order to centralize critical details to send to funders.
Once the alliance activities have been ordered, the next step is to normalize them looking to increase the research program's performance.At this time, the technical and administrative systems of the GAT ecosystem have been implemented, and necessary and regular tasks and activities are executed to accomplish the alliance's goals.Table 4 presents a summary of insights and analysis about the identified variables in the norm/performing stage.

Operations Administration
Management tasks take more time than scientific processes, increasing complexity in the overall ecosystem operation.Proper support of administrative tasks is key to reaching the goal's project.

Coordination
Coordination is the keystone to aligning diverse institutions, each one with its own interests.

Leadership
Even though each project has its own PI, the scientific manager can make decisions that sometimes can be against the Principal Investigators.

Sharing
Due to the effects of COVID-19' on the global economy, new capabilities to share resources between laboratories were developed.
Standardization Some experiments can be executed nowadays in a standard way, with procedures and times controlled.This makes it easier to sell services.

Infrastructure
Some collaborative relationships are based on physical spaces such as specialized laboratories, but these relations can be temporal due to the nature of independence between institutions.

Knowledge transfer Stakeholder turnover
Job turnover considerably affects project performance, especially when one expert or institution leaves.

Networking
During the program assemblage, each project worked as a self-contained project, but now, there is a network of interdependencies and connections between them allowing to exchange of information and aligning processes and outputs.

Alignment
The development phases of each project must be respected but require articulation and synchronization for their entry into operation.Alignment does not fully imply an assemblage between the whole projects, whereas it implies the proper collaboration among them at the necessary level.

Purpose
Collaboration can be based on process capabilities, but also in relation to a more particular object of study, such as a specific plant extract.

Maturity
The level of maturity in the research groups composing the ecosystems determines the effectiveness of collaboration and program performance.

Planning
Evaluation collaboration also implies assessing planning tasks in order to modify them to reduce risk when developing research.

Technology Equipment
In some cases, alignment between team rest on technology availability in the laboratory, for instance, special equipment can be utilized by three programs.

Information systems
The data platforms allow us to perform some experiments and analyses in a more efficient way.The KMS enables information and knowledge sharing among projects and the program governance activities.

Macroeconomy Money
Currency volatility between countries can affect collaboration leading to the development of activities locally to reduce costs.
During the transformation stage, alliances and teams may encounter planned and unexpected events.In this phase, assessing collaboration can help determine whether to improve, restructure, or dissolve it.This leads to three potential outcomes: formal termination, continuation without changes, or modifications to its functioning.Table 5 presents a summary of insights and analysis about the identified variables in the transform/adjourn stage.

Networks Evolution
Collaborative networks to perform the project can derive new alliances to conduct additional investigations.

Alignment
All the Principal Investigators should be fully connected and articulated to get the expected performance.

Collaboration Capabilities
Collaborative work allows for reconfiguring the scientific ecosystem to research new and novel topics that emerged from the program development, but also contingencies related to public health such as COVID-19.

Trust Trust development
The execution of the tasks over time can determine the need to replace a researcher due to engagement.

Institutionality
Engagement Some institutions are invited to make part of the ecosystem but do not contribute as expected.

Mentorship
Less experienced institutions require mentorship at scientific and administrative levels to produce the expected outcomes effectively.

Sustainability
New generations of researchers should be trained in administrative and scientific capabilities to lead large research projects taking into account the lessons learned and best practices derived from the GAT ecosystem.

Funding
Evaluating collaboration also implies assessing support, opportunity, administration, and management from the funding actors and intermediaries.

Phase 4-Assess Levels of Integration
In the CEIF, Woodland and Hutton mention that collaboration relies on integrations between and within organizations, however, the level of integration should meet the purpose and goals of the alliance, because "more integration is not necessarily better" [16].To explore integration in the GAT ecosystem, we customized and applied the Levels of Organizational Integration Rubric (LOIR) [16] to the PIs and researchers of each project in the GAT ecosystem.The LOIR describes five levels of organizational integration (from zero/none to four) and the purposes, strategies/tasks, leadership/decision making, and communication characteristics that tend to be present at each level of integration [16].We asked participants to answer questions about the actual and desired level of integration of each project regarding the GAT ecosystem in each variable of LOIR.We use the levels of collaboration proposed in [27] to categorize the results in terms of networking, coordination, cooperation, and collaboration.We also asked open questions about further and particular actions to increase the level of integration.
Figure 4 shows the quantitative results of the survey.In terms of purpose, whereas the participants consider that they work together to ensure that complementary tasks are carried out (coordination), they expect to form a unified single structure (collaboration).The difference between the current and desired results is −1.56.Regarding strategies and tasks, participants consider that the program develops relationships based on consultancies and there are few complementary goals and tasks (coordination); however, they expect a formal structure to support strategies and tasks enacting a joint mission (collaboration), with a difference of −1.44.In leadership and decision making, participants consider that the proper level should represent centralized leadership and clear decisionmaking mechanisms (cooperation), whereas the current level is based on a non-hierarchical decision-making structure and volunteer leadership.The difference for this category was −1.11.Finally, participants consider that the current level of communication is clear but informal with sporadic conflicts, but they expect to have a more formal communication system.The difference for this category was −1.11.In the leadership/decision-making and communication categories, researchers do not expect to have a full level of integration.Consequently, as the authors of [48] highlight, meaningful integration cannot be simplified; instead, it may call for ineffective interactions to highlight synergies between disciplines.

Phase 5-Assess Cycles of Inquiry
The final phase of the CEIF is about assessing the quality of interprofessional collaboration [16].Effective and efficient interprofessional collaboration rests on individual and team capabilities to collaborate, even if people are not skilled in the practice of collaboration itself.One of the most important questions that emerges when evaluating collaboration in large research groups is whether relationships are leading to the expected organizational performance.Sometimes leaders assume that researchers know how to collaborate, and all the alliances will work very well.In this phase, we use the mapping strategies in Phase 2 to examine the cycles of inquiry in the GAT ecosystem.In doing so, we applied the Team Collaboration Assessment Rubric (TCAR) [16] to 39 out of 115 researchers in the GAT ecosystem.The evaluation is operationalized by exploring four characteristics, dialogue, decision-making, action, and evaluation, at different quantities of attributes and levels of quality (2, 1, and 0).Table 6 presents the results for this phase.
The results show regularity in the scores around 1.65/2.0,where the strongest dimension of collaboration is dialogue (1.70), and evaluation has the lowest score (1.63).There are no significant differences in the results when discriminating between PIs and other researchers, and only slight differences in the dimensions of action and evaluation.The best performance in terms of interprofessional collaboration is in projects 1 and 9, which are related to the ethnobotanic study and clinical trials of P2Et extract, respectively, both directly related to patients.Project 2, the largest and most heterogeneous in the GAT ecosystem, has one of the lowest average scores for the four concepts.In terms of [48], large research groups are more productive than smaller ones, but heterogeneity in disciplines and institutions, together with coordination complexity, can affect performance.

Phase 5-Assess Cycles of Inquiry
The final phase of the CEIF is about assessing the quality of interprofessional collaboration [16].Effective and efficient interprofessional collaboration rests on individual and team capabilities to collaborate, even if people are not skilled in the practice of collaboration itself.One of the most important questions that emerges when evaluating collaboration in large research groups is whether relationships are leading to the expected organizational performance.Sometimes leaders assume that researchers know how to collaborate, and all the alliances will work very well.In this phase, we use the mapping strategies in Phase 2 to examine the cycles of inquiry in the GAT ecosystem.In doing so, we applied the Team Collaboration Assessment Rubric (TCAR) [16] to 39 out of 115 researchers in the GAT ecosystem.The evaluation is operationalized by exploring four characteristics, dialogue, decision-making, action, and evaluation, at different quantities of attributes and levels of quality (2, 1, and 0).Table 6 presents the results for this phase.The results show regularity in the scores around 1.65/2.0,where the strongest dimension of collaboration is dialogue (1.70), and evaluation has the lowest score (1.63).There are no significant differences in the results when discriminating between PIs and other researchers, and only slight differences in the dimensions of action and evaluation.The best performance in terms of interprofessional collaboration is in projects 1 and 9, which are related to the ethnobotanic study and clinical trials of P2Et extract, respectively, both directly related to patients.Project 2, the largest and most heterogeneous in the GAT ecosystem, has one of the lowest average scores for the four concepts.In terms of [48], large research groups are more productive than smaller ones, but heterogeneity in disciplines and institutions, together with coordination complexity, can affect performance.

Summary of Phases Insights
Supporting collaboration can be achieved through technology-based strategies, such as deploying a knowledge management system (KMS) and utilizing online network visualizations, which enable communication and information exchange at the scientific and management levels, facilitating awareness, access, and engagement among researchers.This means that coordination mechanisms are key to promoting collaboration [17].Promoting adequate networking is also a valuable tool to strengthen the GAT ecosystem over time."Knowing who" is the most effective way to coordinate interdependencies between activities and information exchange between people, teams, institutions, and research ecosystems.Therefore, even though collaboration can be highly supported by technology, human relations are the core to leverage it [58].
By monitoring the five stages of development in the GAT ecosystem, it is possible to reveal collaboration issues that can affect the ecosystem's performance.The results suggest that the program aims to move from coordination to collaboration, in a scalable way, emphasizing the need for a unified single structure and formal communication system to support joint missions and decision making.At this point, the GAT ecosystem recognizes the importance of integration of individuals, teams, and institutions conforming to the ecosystem; however, proper levels of integration should meet the specific and particular collaboration requirements of each project, avoiding standardization of levels.Finally, interprofessional collaboration rests on a collaborative culture in which training and support are key variables.

Conclusions
This paper reports the evaluation of collaboration in a large translational research program about cancer treatment.We ground our work on the CEIF.Our qualitative and quantitative analyses suggest that collaboration is crucial for innovation in translational projects, enhancing capabilities by sharing knowledge resources, materials, and expertise.Collaboration drives knowledge production and value creation, and it has become increasingly important in addressing the complex challenges of scientific investigation, leading to increased team-based research output.However, successful implementation requires careful planning and knowledge infrastructure.
We identified several challenges affecting collaboration performance, such as governance, management, processes, and operations, leadership, and social and human relationships.By exploring and analyzing these insights, research project leaders and stakeholders can monitor the progress of the ecosystems and consistently make informed decisions on enhancing the performance of the collaboration.Our insights also lead to considering that evaluating collaboration implies not only quantifying the outputs based on traditional metrics such as the number of papers, conferences, and so on.Evaluation attempts to analyze in a holistic way how leaders can better guide the research projects in order to reach cohesion, synergy, and cooperation among stakeholders.Additionally, the process we conducted, and the corresponding results can also guide researchers when designing a new research ecosystem, aiming at sustainability in a current one, or even deciding when it should be adjourned.All of these decisions are valuable information for the research ecosystem performance evaluation.
This study demonstrates that technology-based strategies, networking, a systematic evaluation of collaboration stages, integration, and interprofessional collaboration are crucial in promoting effective collaboration in a translational research program.These findings provide valuable insights for managing and improving collaborative efforts in large research groups.The results allow us to anticipate collaboration issues with actionable and opportune strategies that can enhance the planning process, ecosystem performance, sustainability, research outcomes, profit, and the program's overall success.
The novelty of this study rests on evaluating the collaboration conditions in translational research projects leading to sustainability, as posited in [62], of the services developed during the execution, as well as identifying reproducibility conditions of this initiative in other disciplines and sectors different from cancer research.
The findings of this paper are specific to translational research programs and cannot be directly applied to other sectors.The generalizability of these findings will require investigation in other private or public sectors and with a different unit of analysis.Reproducibility studies in other research ecosystems currently being executed in Colombia in sectors such as sustainable energy, sustainable agricultural crops, productive and social inclusion, and others, could potentially lead to the generalization of collaboration factors identified in this study.These further studies could also promote adjustments in public policy, enabling scientific collaboration.
Even though the GAT ecosystem involves international actors, we focused on local actors, because cultural differences in collaboration research can generate bias: future studies could explore the multinational aspects related to collaboration in translational research.During the four stages of development in Phase 3, we identified several variables impacting collaboration performance that merit further analysis in order to determine the relative impact of each one.Incorporating this broader perspective in future studies would capture the diverse dynamics of collaboration in large research programs.

Figure 1 .
Figure 1.The KMS for the GAT ecosystem.

Figure 1 .
Figure 1.The KMS for the GAT ecosystem.

Figure 2 .
Figure 2. The visualization map for the GAT ecosystem.

Figure 2 .
Figure 2. The visualization map for the GAT ecosystem.

Figure 3 .
Figure 3. Network visualization of the GAT ecosystem.

Figure 3 .
Figure 3. Network visualization of the GAT ecosystem.

Figure 4 .
Figure 4. LOIR results for the GAT ecosystem.

Figure 4 .
Figure 4. LOIR results for the GAT ecosystem.

Table 1 .
Evaluation models for research collaboration.

Table 6 .
TCAR results for the GAT ecosystem.

Table 6 .
TCAR results for the GAT ecosystem.