Implementing Open Innovation in Research and Technology Organisations: Approaches and Impact

Abstract

Research and development organisations (RTOs) are at the heart of innovation systems. They help to connect innovation system actors to foster industrial innovation. Due to this intermediary role, they act as paradigmatic open innovation (OI) actors. In this context, RTOs need to balance their knowledge stocks and flows, while assuring their own innovation capabilities and positively impacting the innovation system they influence. Thus, RTOs need to develop collaboration approaches that support their own performance while increasing their capabilities and not threatening their competitive advantages derived from their knowledge stock. In this study we extend the OI research to research organisations analysing their OI approach and the impact on its own performance, developing a new framework for OI study in RTOs using a partial least squares structural equation modelling (PLS-SEM) approach. The research, based on a sample of Spanish RTOs, arises two substantive conclusions. First, an increased number of collaboration partners (collaboration breadth) and the use of a variety of OI practices have a strong impact on RTOs’ overall performance (scientific, transference, and economic results). Second, RTOs need to foster their aperture and promote an active management to benefit from collaborating partners, whereas managers should pay special attention to questions related to the protection and management of intellectual property when promoting the use of different OI practices. Both results also emphasise the importance of managing firms’ knowledge flows in the implementation of OI in RTOs.

1. Introduction

The collaborative approach to science, technology and innovation has become very important for innovation and technology diffusion and has, therefore, caught the attention of managers, economists, and politicians [1]. This way of organising research, development, and innovation activities is an effective way of sharing knowledge, obtaining complementary assets and generating new technologies in most sectors where technological advances are rapid [2].

From the perspective of innovation, Chesbrough [3] coined the term open innovation (OI) to describe a trend in innovation theory, evolving from a closed to an open approach. He defined OI as “the use of knowledge flows from abroad and outward to accelerate the innovation process and expand the market for the external use of innovation, respectively”. The “open” approach is being adopted in the innovation policies exemplified in Europe with the “open science, open innovation, open to the world” policy [4].

All these open approaches to science and innovation share a common underlying understanding: that the collaboration and sharing approach will provide benefits to the advance of science and to the translation of science in innovations [5] that will fuel economic growth [6]. From the perspective of individual organisations, one of the main claims of the OI approach is that organisations applying it can improve their innovative performance [7,8,9,10].

Research and Technology Organisations (RTOs) are non-profit, industry-oriented, research and development organisations, with the main mission to drive business innovation through technology transfer, especially in small and medium enterprises (SMEs) [11] acting as a catalyst in the innovation system. RTOs are characterised by their role as intermediaries in innovation systems, filling the gap between scientific and technological knowledge and innovation [12].

RTOs as intermediaries are already connected to different actors in the innovation systems; however, several external forces make them pursue more connected strategies with different types of actors and use different knowledge transfer practices [13]. First, due to pressures in public R&D funding, they are required to perform and find diversified funding sources [14]. Second, many R&D funding programs require the formation of project consortia, including different actors [13]. Third, more companies are embracing the OI approach; therefore, seeking knowledge from external actors and demanding more collaboration from RTOs. Fourth, technology development is becoming more multidisciplinary, complex, and dynamic, so it is very difficult for a single organisation, especially smaller ones, to possess the necessary capabilities [15]. These tendencies push RTOs towards a more open and collaborative approach to research and innovation.

From the RTOs perspective, an important question arises in this context: do RTOs benefit from adopting a more open approach to innovation? This is very relevant to RTOs, as they also need to perform successfully to sustain themselves and continue to develop their mission. Contrary to the commercial firms, in which innovative success is measured by metrics like innovations or financial benefits, the success of RTOs is measured by different performance metrics [16,17]. Thus, RTOs need to perform successfully in a number of areas to ensure their sustainability: (1) knowledge generation, to ensure they generate new relevant knowledge and to remain at the forefront of technology research [18]; (2) knowledge and technology transfer, to provide companies with knowledge and technology as their main mission [19], and (3) economic results, that supports their operation and long term sustainability [20].

The main mission of RTOs is to transfer knowledge and technology to companies, and, therefore, it seems relevant to focus on measuring this impact [21] from an OI perspective. However, despite the RTOs position in the innovation system, which suggests a natural OI role in knowledge generation and transfer [14], there is a significant gap in the literature in the analysis of OI in RTOs [22,23]. Academics have mainly studied RTOs indirectly as collaborators and partners of different types of private firms in an OI context, rather than as the beneficiaries of the OI paradigm [24].

Therefore, this research tries to address this gap exploring OI in RTOs, evaluating the effect that an OI approach has on the performance of RTOs and analysing the relevant managerial and organisational factors that affect their OI approaches. The research answers the call to extend the OI study to other types of organisations different from commercial firms [25]. Thus, the research contributes to the advance of the knowledge of OI in different areas:

First, we contribute to extending the OI approach in a new context, which has been very scarcely studied from the OI perspective [13,23], i.e., RTOs that are a non-profit research organisation with an intermediary role in the innovation systems.

Second, we contribute to the literature of intermediaries in the innovation systems, focusing on them, and studying the effect that OI has on their own performance, instead of the partner role, which has been the dominant perspective [11,26,27].

Third, the research contributes to increase our understanding of how the OI approach can be applied in the practice of RTOs, paying attention to OI enablers in management and organisation relevant to RTOs.

Fourth, the study provides empirical evidence of the OI approach in Spanish RTOs and evaluates its effect in RTOs performance. The performance of RTOs, contrary to the typical measures of innovative performance used in companies: i.e., innovative products and services sales is measured by a composite index that considers the RTOs core activities: knowledge generation, technology transfer, and economic results that are necessary for successful and sustainable RTOs.

Additionally, the research provides insights for RTOs managers and stakeholders on the effect of the organisational and management aspects in the adoption of a successful OI approach.

The empirical study will be based on the Spanish population of RTOs, based on a self-administered survey directed to RTOs’ managers. We will use a PLS-SEM approach [28] for the study that provides the required flexibility and tools to explore complex relationships among different OI dimensions and performance. Thus, this research provides a fresh, exploratory perspective of OI in RTOs, which differs from the traditional OI studies in commercial firms in two main areas: the R&D nature and intermediary role of RTOs on the innovation systems that suggest a natural openness and their special mission, with a non-profit character, that conditions their management and key performance metrics [16].

The paper structure is as follows: the next section contains a literature review of the advances and research developed about the impact of OI in the performance of an organisation, to bring key factors and dimensions to the context of RTOs. Section 3 describes the research framework and hypothesis, while Section 4 includes the data collection scheme used. Section 5 explains the research methodology followed in accordance to the research framework established, as well as the PLS-SEM modelling and results. The discussion of research results and hypothesis validations are presented in Section 6. Finally, the main conclusions, limitations, and implications of the research are stated in Section 7, suggesting some future research avenues.

2. Literature Review

OI is a very rich concept, which can be approached and implemented in several different ways [29]. In the literature, the concept of “openness” has been understood and measured in different ways, which resembles the multiple dimensions of OI [30]. Thus, to study OI in RTOs, we will consider several key OI aspects taken from the OI literature to develop our research framework and corresponding research question and hypothesis.

2.1. Collaborating with External Partners: Collaboration Breadth and Depth

Companies long ago recognised that relations with other agents and engaging in cooperative partnerships could provide many benefits in R&D, innovation, and technology transfer [2,31,32].

Several authors have tried to analyse the impact that each type of partner has on the organisation’s performance [33]. Belderbos et al. [34] considered four basic types: competitors, customers, suppliers, and knowledge agents (Universities/RTOs) and observed that they have differentiated effects on performance. Other authors have broadened the typology of agents, introducing consultancies or intermediaries [7], business and trade associations [35], or companies from other sectors [36]

The influence of simultaneous collaborations with these four types of agents [9] has also been analysed, with different effects depending on the size of the company and the specific combination of types of agents.

Two widely used measures of collaboration with external partners were introduced by Laursen and Salter [37] to measure the external knowledge search strategy: the breadth of the search measured the number of external sources of knowledge used, and the depth of the search that measures the intensity of the use of those external sources. Later on, the concept of search breadth and depth was extended to collaboration breadth and collaboration depth to consider the double direction of the knowledge flow [38].

2.2. OI Modes and Practices

In addition to the research studies on cooperating partners, OI research literature introduces three different OI modes [39] regarding the direction of knowledge flow: inbound, outbound, and coupled. These modes influence the mechanisms or practices to acquire, transfer, or co-develop knowledge, respectively, in different OI practices.

The inbound mode has been the most studied in the literature, as mentioned by several authors [9,40], while the outbound and coupled modes have been far less considered [41,42]. Ahn et al. [7] have studied the effect of different inbound modes, finding a positive effect of collaborative R&D projects, user involvement, and technology acquisition on performance. In relation to inbound mode, a review study [43] evaluated the impact on companies of the use of external sources of knowledge (Universities, Technology Centres, and knowledge-intensive business services). The study found that larger, more R&D-intensive companies and high-tech companies are more likely to use external knowledge and that the use of that knowledge is associated with better innovation performance.

The outbound mode refers to the external technology commercialisation [42] to leverage internal R&D efforts [44] and tends to provide benefits for organisations as revenue generator [7,42,45]. However, the outbound mode can also provide risks [46], i.e., knowledge losses or high costs, that require a well organised R&D portfolio management approach [42].

Finally, and complementing both approaches, some authors have considered the three modes of OI altogether. For example, Cheng & Huizingh [47] found that the application of the three modes of OI significantly and positively impacted four dimensions of innovative performance: new product/service innovation level, new product/service success, customer performance, and financial performance, although each type of OI mode had a different intensity of impact. These authors argue that entrepreneurial orientation, with associated proactive and entrepreneurial processes, create a fertile environment for OI. Other authors [45], also found a variety of impacts from the use of different modes of OI and their different practices. Specifically, acquisitions, out-licensing, co-patents, and manufacturing alliances appeared to impact positively on both new products and revenues.

2.3. Organisational and Managerial Issues in OI

As several authors showed, active management is connected to OI application and performance [47,48]. The implementation of OI strategies requires a specific culture in the organisation that values external competences [49], which differs from those applying closed innovation strategies. In this sense, an exemplary problem is employees’ negative attitudes against external knowledge: the syndrome of not invented here (NIH), and against the external exploitation of the assets of own knowledge: the no shared here (NSH) syndrome [50,51].

Organisations are professionalising the internal processes to manage OI more efficiently and effectively [48,49] since it has been shown that OI management influences the effect of OI in performance [52].

When applying OI, organisations open the flow of knowledge into and outside the organisation. However, at the same time, they need to protect intellectual property in order to capture the returns of their innovation efforts, which gives rise to the “paradox of openness” [53]. Thus, intellectual property management and protection acquires great relevance [54] to ensure that firms capture the value of their technologies [55], and for this, the management of intellectual property rights (IPR) becomes an element of great relevance, especially in an environment of multiple and varied collaborations [56,57] in which RTOs work.

2.4. Open Innovation and Performance in RTOs

The literature of OI in RTOs, and specifically, the studies that research the impact of OI in the performance of RTOs (or other types of research organisations) is very scarce. Many authors approaching it focus on the impact of collaboration rather than a more holistic view of OI [11,26,27]. Some other authors have analysed (focused on company-owned research centres) the effect of inbound OI on the performance of collaborating company-owned business units and research centres; finding a positive relationship of inbound OI and performance for both business unit and research centre [58]. In the same line, Asakawa et al. [59] analysed the impact of OI with the same dual perspective (company and research centre) finding that research labs increase their performance by the adoption of OI.

Other authors analysed the effect of different collaborations profiles and their effect on the scientific performance of R&D institutes measured by publications related indicators [60]. In the same vein, Chen et al. [22] studied the effect of the position of research centres in research collaboration networks with industries and/or universities and their impact on their scientific performance. De Silva et al. [23] analysed the impact of knowledge-based practices used in collaborations with external agents on the internal value creation in RTOs to ensure their long-term sustainability.

However, despite the RTOs position in the innovation system that suggests a natural OI role, there is a significant gap in the literature in the analysis of OI in RTOs, and specifically on how it affects performance [22,23]. This is very relevant for RTOs since the current tendency towards openness pursued by different actors in the innovation system urges them to embrace a more conscious OI approach [27]. However, the lack of knowledge of the effect of different OI dimensions (collaborating partners and OI practices) in RTOs’ performance and the extent of the necessary organisational and managerial skills, may hinder their adoption or lead to non-efficient OI adoption. Therefore, there is a need to shed some light in this phenomenon, obtain evidence on the role of OI in RTOs, and provide some guidelines to RTO stakeholders to improve performance, and as a result, enhance their contribution to industrial innovation and the regional economic development.

3. Theoretical Framework and Hypothesis

In order to clarify the approach, benefits, and threats of the implementation of OI in RTOs, our research has focused on the analysis of the effect of OI collaboration schemes and practices on RTOs’ overall performance (scientific, transference, and economic results), as well on the analysis of the associated managerial and organisational issues.

We have developed a new research model, that besides integrating previous authors approaches studying the effect of OI in performance of firms [10,41,45,61], also considers and integrates the special characteristics of RTOs and their features regarding their role in the innovation systems. This newly defined model helps to understand the phenomenon of OI in RTOs from an exploratory and more comprehensive approach, combining previous research efforts that studied partial aspects of OI in RTOs [11,26,27].

This approach will allow us to explore the relations of different OI measures, answering the following research questions: (1) which is the effect of OI in the performance of the RTOs? (2) Which is the effect of organisational and management factors in OI application in RTOs?

To answer those questions, the research model includes several OI measures that are relevant for RTOs, as shown in Figure 1. The model basis considers: on the one side, the two main dimensions of the OI paradigm, the collaborating partners (measured by collaboration breadth and collaboration depth) and the OI practices (measured by OI practices) that considers the three main modes of OI: inbound, outbound, and coupled with its associated practices, and analyses its effect in RTOs’ overall performance (RTOs’ performance). On the other hand, to explore how other managerial and organisational aspects affect the application of OI in RTOs, we include the organisational commitment towards OI (measured using three variables: organisational aperture, OI management, and IPR protection) to analyse its effect in the OI application in RTOs.

Due to the singular nature of RTOs, their underlying business model relies on achieving results in a set of different dimensions [16,17,19,62] that contribute to their overall performance and to achieve their long-term sustainability. Thus, RTOs are expected to provide results in three dimensions: on the knowledge generation dimension, their research activities need to contribute to the knowledge advance maintaining them at the forefront of technology research and increasing their knowledge stocks for future technology transfers [18]. On the technology transfer dimension, according to their main mission, they impact the companies by transferring knowledge and technologies [19]. Finally, on the economic front, even if they are a non-profit organisation, they need to obtain and efficiently manage their economic resources to ensure their operations [20]. We argue that collaborating with external partners and using different OI practices can benefit the different performance dimensions of RTOs, thus positively affecting their overall performance.

Regarding collaboration with external partners, several studies have shown a positive effect of the use of external sources of knowledge in the performance of firms [7,8,9,10]. However, cooperation with external agents has associated costs, and in some cases, the results obtained from external collaboration may decrease [37,63,64]. This effect may be related, among other factors, with the internal R&D capacity, as some authors found that companies with large R&D capabilities are better able to capture and exploit external knowledge [65]. Interestingly, RTOs may be better positioned than firms to benefit from collaborating from external partners since they possess higher internal R&D capabilities and absorptive capacity that facilitate the use of external knowledge [66].

When collaborating with external partners in innovation activities, RTOs need to decide their approach in terms of broader or deeper collaboration strategies. RTOs that develop broader and deeper collaboration strategies may be able to increase their knowledge generation and transfer, thus improving their capabilities to generate industrial innovation. Thus, to explore the influence of broader and deeper external collaboration in RTOs’ performance, we use collaboration breadth and collaboration depth variables [37,67]. Collaboration breadth refers to the number of external partner types that collaborate with RTOs in innovation-related activities, whereas collaboration depth refers to the intensity of these collaborations.

Previous studies found that research institutes scientific performance could benefit from their interaction with an ample number of actors in the triple helix innovation systems [22]. Companies increasingly pursuing OI strategies can benefit by strengthening their collaboration with RTOs [26], thus also increasing the technology transfer impact of RTOs. Additionally, increasing the collaboration with multiple partners in government-funded programs and increasing collaboration with companies may allow them to diversify their funding sources, improving their economic results. Therefore, we expect that collaborating with multiple agents in the innovation systems, both broadly and deeply, may allow RTOs to increase their knowledge stocks and knowledge transfer, improving their overall performance. Therefore, we put forward the following hypotheses:

H1. 

Collaboration breadth has a positive effect on the overall performance of RTOs.

H2. 

Collaboration depth has a positive effect on the overall performance of RTOs.

Regarding the effect of the use of multiple OI practices in the main OI modes: inbound, outbound, and coupled, several studies have found that using a variety of OI practices have a positive effect performance of firms [45,63]. RTOs, due to their intermediary role in the innovation system practice, the three main modes of OI: inbound, acquiring knowledge from other agents like universities, for example, inviting external personnel or developing collaboration agreements for knowledge acquisition; outbound, transferring their internal knowledge to other partners by means of sale of know-how or knowledge-based services, and finally, coupled, co-developing knowledge with other partners, for example, in collaborative R&D projects. For RTOs, the use of a wide variety of OI practices could open new opportunities for knowledge acquisition, creation, and transfer, providing them with benefits in their overall performance. For example, the use of more technology transfer practices, like the sale of know-how or the creation of spin-offs will provide them more opportunities for benefiting from their internal knowledge assets, or developing collaboration agreements with other knowledgeable agents will allow them to gain the capacity to accelerate their knowledge and technology development. Thus, we expect that the use of different OI practices will have a positive impact on the performance of RTOs and therefore put forward the following hypothesis:

H3. 

A wider use of OI has a positive effect on the overall performance of RTOs.

4. Data Collection

The study focuses on Spanish RTOs, and its population is based on the Spanish Register of Innovation and RTOs (CIT) of the Ministry of Science, Innovation, and Universities [68]. As of 15 January 2018, the CIT registry had 63 entities registered as RTOs. Once the three inactive entities were discarded, the final population of 51 RTOs was obtained by selecting those oriented mainly to sectors of medium or high R&D intensity [69], discarding those mainly oriented to sectors of low R&D intensity, fundamentally RTOs oriented to the primary sector.

The data has been collected based on a self-administered survey, which has been designed to maximise motivation and facilitate respondents to respond accurately, trying to minimise methodological bias [70]. In the design of the survey, many measurement scales and questions previously validated in the OI and RTOs literature have been used.

The survey, aimed at RTOs’ managers, uses a Likert scale of 1 to 5 and includes four sections: (1) the collaboration network (breadth and depth); (2) the use of OI practices; (3) the organisation and management issues (organisational aperture, the OI management, and IPR protection), and (4) the RTOs’ performance.

After an initial validation of the survey through personal interviews with two directors of RTOs and the consequent adjustments, the final survey was sent, and responses were received from May to September 2018. Finally, 37 responses were obtained, which implies a ratio of 73% of the total population selected, which can be considered very representative.

The sample has an important degree of reliability since 89% of the responses come from first level management (being 43% of the total General Directors) and where 84% of the total report more than 10 years of experience. On the other hand, the sample has a good geographic distribution since all regions with more than one RTO are represented with at least 50% of the regional RTOs. In addition, in order to detect the common methodological bias, the one-factor Harman test was carried out [70]. This test has shown that the main factor explains only 26.21% of the variance, well below 50%, which indicates that no evidence of common methodological bias has been found [70]. Additionally, evaluations of potential biases have been carried out using the T-Student statistical test [71]. First, we analysed the non-response bias, which consists of comparing the results of early respondents with late respondents and analysing the differences [72], and no significant differences were found between early and late respondent’s groups. Secondly, it has been verified whether the profile of the respondent (general manager, manager, or others) influences the data obtained in the sample. No significant differences were found between the different profiles surveyed.

With all these analyses, we conclude that the data obtained are of sufficient quality and adequately reflect the population of selected RTOs.

5. Research Method and Results

5.1. A PLS-SEM Modelling Approach

To answer the research questions and test the proposed hypothesis, partial least squares structural equation modelling (PLS-SEM) [28] was applied using the SmartPLS software v3.2.8 [73]. This study aims to explore the OI approach and the performance of RTOs, and the variance-based PLS-SEM approach is particularly beneficial due to the exploratory nature of the research and the complex interactions involved [28,74]. The great flexibility it provides in the modelling of variables, from a single item to complex variables, such as OI practices or RTOs’ performance using reflective and formative constructs [75] or higher order constructs [76], is another important reason. Furthermore, PLS-SEM has been used in previous studies [58,77,78,79], and the fact that it is based on a series of ordinary least squares regressions and its statistical power makes it suitable for a relatively small sample and population sizes.

PLS-SEM methodology contemplates the modelling of two elements [28]: the measurement model and the structural model. The measurement model represents the relationship between the constructs or variables and the items used for their measurement, while the structural model represents the relationships among variables or constructs. In PLS-SEM model evaluation and validation are carried out in two stages: first, the measurement model is evaluated, where the constructs used to measure the variables are validated. In a second stage, the structural model is evaluated, where the relationships between the different variables are analysed, fundamentally, their magnitude and significance, allowing hypothesis testing.

In the next sections, we present the PLS-SEM study that follows the recent analysis and reporting recommendations made in the literature [28,80,81].

5.2. Measurement Model

The measurement model includes first order reflective constructs and second order reflective-formative constructs [76] to ensure the most appropriate modelling for all variables. The selection of each measurement model considered theoretical issues as well as suggested selection criteria [82].

The evaluation of the measurement model is made according to the specific model used for each variable, either formative, reflective, or second order construct. The evaluation process, as described in next sections, starts with the first order construct variables assessment and continues with the second order constructs evaluation, thus fully validating the measurement model.

5.2.1. Variables, Survey Items, and Constructs

Collaboration breadth and collaboration depth operationalisation [38,67] is shown in

Table 1. Collaboration breadth and depth constructs survey questions.

Collaboration Breadth and Collaboration Depth
Survey items/Data (Likert scale: 1 Very low–5 Very high)	Authors
Collaboration level with each partner type (total of 24 items: 8 partner type at 3 geographical levels: regional, national, and international):	[7,54,67,83,84]
1. Universities
2. Public Research Centers
3. Research and Technology Organisations
4. Consulting and related services firms
5. Scientific-technical networks and associations platforms
6. Business network and associations
7. Companies
8. Public administrations

. It is inspired by the original Laursen and Salter [37] breadth search and depth search concepts measured as the counter mode. Thus, the operationalisation is carried out counting the collaboration with 8 types of entities (since the survey provided data for each partner type at three geographical levels regional, national, and international, we use a total of 24 items), both for the variable collaboration breadth (counting each collaborating entity: item ≥ 2) and for the variable collaboration depth (counting each collaborating entity with deep interaction: item ≥ 4). Thus, these variables range from zero to 24. Relevant types of partners have been considered based on the specific nature of RTOs, extending the ones identified by previous researches focused on the study of OI in firms [7,54,67,83,84].

Since collaboration breadth and depth are count type variables, they enter directly into the PLS-SEM model as single item constructs. PLS-SEM cannot evaluate single items for validity, and therefore we consider them valid as they have been widely used in the OI literature [7,9,37,40,53,54,64,67,83,84,85].

OI practices are grouped into the three different modes of OI described in the literature: inbound, outbound, and coupled [39] according to the main direction of the flow of knowledge: towards the organisation, out of the organisation, and in both directions respectively. Each mode is materialised with several OI practices already analysed in previous studies and relevant for the study of OI in RTOs. These three modes present three different dimensions of OI practices [45] and are, therefore, modelled as a formative element, since they form a set of complementary factors, which may be independent of each other, forming the OI practices composite indicator [28,86]. Each OI mode is modelled, as the first order reflective indicator with a set of reflective variables. Thus, obtaining the so-called second order reflective-formative model [28,76] in Figure 2. The three OI modes are measured by a set of survey items, as shown in

Table 2. Open innovation (OI) Practices survey questions.

OI Practices
Survey items/Data (Likert scale: 1 Very low–5 Very high)	Authors
Use of the following practices in the last three years (11 items):	[7,20,41,47,55,87,88,89,90]
Inbound
(PR1) Technology acquisition (buying)
(PR2) Formal collaboration agreements with knowledge agents
(PR3) Personnel exchange
Outbound
(PR4) Sale of know-how
(PR5) Creation of Spin-offs
(PR6) Marketing agreements for RTO’s own technologies
Coupled
(PR7) Shared patents with other organisations
(PR8) Collaborative projects with regional public funding
(PR9) Joint doctoral theses
(PR10) Informal interactions with other organisations
(PR11) Participation in conferences and fairs

.

Regarding RTOs’ commitment towards OI, the variables, organisational Aperture, OI management, and IPR protection are operationalised (see

Table 3. Organisational aperture, OI Management, and intellectual property rights (IPR) Protection survey questions.

Organisational Aperture
Survey items/Data (Likert scale: 1 strongly disagree–5 strongly agree)	Authors
Agreement level with (4 items):	[77]
(A1) The RTO has a culture of encouraging external collaborations
(A2) The RTO has a predisposition to share experiences through collaboration
(A3) The top management of the RTO is proactive in the collaboration with external entities
(A4) In general, the RTO relies on external partners
OI Management
Survey items/Data (Likert scale: 1 strongly disagree–5 strongly agree)	Authors
Agreement level with, (6 items):	[44,91]
(G1) The open innovation strategy is documented
(G2) Responsibilities for open innovation are evaluated periodically
(G3) There are written procedures and rules on open innovation
(G4) There are formal processes for selecting partners (typology and specific partners)
(G5) Analyse the objectives and risks of collaborations
(G6) The results of collaborations are measured and evaluated periodically
IPR Protection
Survey items/Data (Likert scale: 1 strongly disagree–5 strongly agree)	Authors
Agreement level with (3 items):	[56,92]
(I1) Degree of protection of technologies through intellectual property rights, especially patents
(I2) The role of intellectual property in the RTO’s strategy
(I3) Importance of intellectual property in technological transactions

) by means of reflective elements as shown in Table 3 since the scales used for their respective measurement items that form it are interchangeable and the causal priority goes from the construct to the indicator [28].

Organisational performance is broadly recognised as a multidimensional concept and is considered as a paradigmatic example of a formative indicator [28,93]. RTOs due to their nature are expected to provide results in several dimensions: on the scientific dimension they generate new knowledge, measured mainly with publications and patents [18]; on the technology transfers dimension, they impact the companies by transferring knowledge and technologies [19]; finally, they need to perform economically to ensure their sustainability [20]. Thus, we developed a new Performance construct, modelled as a formative element, since they form a set of complementary factors, which may be independent of each other, forming the RTOs’ Performance composite indicator with these three dimensions: Scientific, Transference, and Economic results. This second order reflective-formative model [28,76] uses items previously used in other researches that are suitable for our research approach (

Table 4. Performance survey questions.

Performance
Survey items/Data (Likert scale: 1 Much worse–5 Much better)	Authors
Evolution of performance in the last 3 years (9 items):	[16,18,19,45,62,87,94]
Scientific results
(RE1) Scientific-technical publications
(RE3) Completed doctoral theses
(RE2) Filled patents
(RE4) Patents transferred to the market
Transference results (results transferred to companies)
(RE5) New innovations for the company
(RE6) New innovations for the market
Economic results
(RE8) Revenues obtained in the market
(RE9) Billing per employee
(RE7) Self-financing capacity

).

The resulting PLS-SEM model is shown in Figure 2.

5.2.2. Construct Evaluation Method

Due to the use of second order variables in the model and their endogenous nature, we had to use a combination of approximations to both variable and model evaluation using a two-step method [95]. In the first step, variable evaluation, we used the repeated indicators approach [76,96] in which the items of the first level reflective variables also load in the second level. In the second step, structural model evaluation, we used the single item variables calculated from the first step.

The repeated indicators method [76] was used for modelling the second order variables OI practices and performance, using Mode B settings for items loading directly in the second level constructs (formatively) and mode A for the formative variables composing the second level constructs. We also used mode A [97] for the rest of first-level variables with a reflective character. Regarding the path method, the path weighting scheme was used [28].

5.2.3. First Order Constructs Evaluation

To evaluate the first-level reflective variables of the model, including first order variables and the first level of second order variables, we followed Hair et al. [28] evaluating: (1) reliability, (2) convergent validity, and (3) discriminant validity.

The reliability of the internal consistency of reflective constructs is based on two criteria: (1) Cronbach’s alpha and (2) composite reliability, as shown in

Table 5. Reliability and convergent validity of reflective variables.

Latent (Reflective) Variable	Items	Convergent Validity			Reliability of Internal Consistency
		Loads	Indicator Reliability	AVE	Composite Reliability (CR)	Cronbach’s Alpha
		>0.7	>0.5	>0.5	0.6–0.9	0.6–0.9
Organisational Aperture	A1	0.945	0.893	0.810	0.944	0.922
	A2	0.941	0.885
	A3	0.810	0.656
	A4	0.898	0.806
IPR Protection	I1	0.881	0.776	0.752	0.901	0.836
	I2	0.901	0.812
	I3	0.818	0.669
OI Management	G1	0.894	0.799	0.768	0.952	0.939
	G2	0.911	0.830
	G3	0.921	0.848
	G4	0.856	0.733
	G5	0.808	0.653
	G6	0.866	0.750
Inbound	PR3	0.849	0.721	0.621	0.830	0.695
	PR1	0.704	0.496
	PR2	0.804	0.646
Outbound	PR4	0.773	0.598	0.609	0.824	0.685
	PR5	0.768	0.590
	PR6	0.800	0.640
Coupled	PR7	0.768	0.590	0.550	0.859	0.796
	PR8	0.711	0.506
	PR9	0.756	0.572
	PR10	0.740	0.548
	PR11	0.734	0.539
Scientific results	RE1	0.772	0.596	0.640	0.880	0.820
	RE2	0.847	0.717
	RE3	0.860	0.740
	RE4	0.733	0.537
Transference results	RE5	0.969	0.939	0.945	0.972	0.942
	RE6	0.975	0.951
Economic results	RE7	0.834	0.696	0.732	0.891	0.818
	RE8	0.862	0.743
	RE9	0.872	0.760

.

The values obtained for Cronbach’s alpha exceed the value of 0.7 [81] for all values except for the inbound (0.695) and outbound (0.685) variables. These two values exceed the value of 0.6 admitted for exploratory research [28,81] and, therefore, we consider them valid. Similarly, the composite reliability (CR) of all constructs far exceeds the recommended value of 0.7 [98] and, therefore, validates the reliability of the internal consistency of constructs.

Convergent validity is checked in two ways: (1) by analysing the loads and reliability of the items and (2) by the average variance extracted (AVE). Indicator loads in constructs exceed the recommended value of 0.7 [80,98] in all cases. Similarly, the reliability of the indicator, which is the square of the loads, also exceeds the recommended value of 0.5 [28]. The value of the AVE shows in all constructs a value higher than 0.5 [81,98], thus verifying the convergent validity of the constructs.

To evaluate discriminant validity, we used three methods: (1) Fornell–Larckert criterion, (2) cross loads, and (3) the HTMT (Heterotrait–Monotrait) method. The Fornell–Larckert criterion states that the AVE from each construct must be greater than the squared correlation with the rest of the constructs.

Table 6. Discriminant validity. Fornel–Larckert criteria.

	Collaboration Breadth	Organisational Aperture	Coupled	OI Management	IPR Protection	Transference Results	Inbound	Outbound	Scientific Results	Collaboration Depth	Economic Results
Collaboration breadth	1.000
Organisational Aperture	0.515	0.900
Coupled	0.249	0.318	0.742
OI Management	0.442	0.295	0.222	0.877
IPR Protection	0.378	0.451	0.519	0.336	0.867
Transference results	0.314	0.060	–0.001	0.465	0.324	0.972
Inbound	0.381	0.220	0.415	0.192	0.317	0.206	0.788
Outbound	0.305	0.164	0.539	0.387	0.620	0.332	0.441	0.781
Scientific results	0.292	0.215	0.548	0.376	0.475	0.125	0.149	0.264	0.805
Collaboration depth	0.618	0.527	0.492	0.349	0.399	0.225	0.524	0.535	0.036	1.000
Economic results	0.436	0.000	0.179	0.159	0.264	0.397	0.343	0.424	0.223	0.415	0.856

shows the results of the Fornell–Larckert criterion, where the diagonal values (in bold) show the square root of the AVE, and the values below the diagonal show the estimated correlation between the factors. The values obtained show that the correlations between variables are lower than the AVE and, therefore, show discriminant validity [81,98].

The second method of assessing discriminant validity is the analysis of cross loads of items in constructs. As shown in

Table 7. Items cross loads.

Items	Organisational Aperture	OI Management	IPR Protection	Inbound	Outbound	Coupled	Scientific Results	Transference Results	Economic Results
A1	0.945	0.318	0.432	0.227	0.188	0.370	0.210	0.018	−0.018
A2	0.941	0.261	0.366	0.132	0.124	0.224	0.092	0.067	0.000
A3	0.810	0.163	0.333	0.116	0.121	0.365	0.263	0.062	−0.152
A4	0.898	0.289	0.483	0.299	0.149	0.214	0.246	0.078	0.123
G1	0.361	0.894	0.305	0.189	0.382	0.179	0.244	0.346	0.027
G2	0.275	0.911	0.290	0.216	0.436	0.157	0.281	0.488	0.200
G3	0.362	0.921	0.240	0.21	0.357	0.207	0.186	0.443	0.041
G4	0.090	0.856	0.364	0.189	0.338	0.287	0.427	0.434	0.053
G5	0.176	0.808	0.248	0.001	0.252	0.181	0.423	0.376	0.202
G6	0.264	0.866	0.323	0.173	0.270	0.166	0.436	0.361	0.299
I1	0.326	0.275	0.881	0.256	0.553	0.433	0.412	0.378	0.369
I2	0.463	0.209	0.901	0.276	0.553	0.582	0.591	0.131	0.241
I3	0.389	0.428	0.818	0.301	0.504	0.306	0.183	0.354	0.030
PR1	0.046	0.109	0.194	0.704	0.368	0.153	−0.174	0.283	0.397
PR2	0.418	0.253	0.306	0.804	0.347	0.398	0.279	0.184	0.165
PR3	0.028	0.083	0.239	0.849	0.340	0.389	0.171	0.056	0.288
PR4	0.046	0.287	0.54	0.238	0.773	0.41	0.315	0.247	0.155
PR5	0.103	0.245	0.466	0.392	0.768	0.579	0.304	0.145	0.463
PR6	0.253	0.396	0.443	0.392	0.800	0.213	−0.051	0.428	0.341
PR7	0.194	0.360	0.551	0.273	0.540	0.768	0.499	0.193	0.338
PR8	0.233	0.014	0.425	0.192	0.346	0.711	0.341	−0.042	0.131
PR9	0.076	0.064	0.323	0.413	0.436	0.756	0.577	−0.089	0.145
PR10	0.360	0.167	0.305	0.308	0.344	0.740	0.291	−0.154	0.075
PR11	0.348	0.194	0.309	0.337	0.306	0.734	0.284	0.071	−0.060
RE1	0.119	0.280	0.353	0.088	0.141	0.287	0.772	0.143	0.376
RE2	0.190	0.115	0.421	−0.005	0.174	0.488	0.847	0.120	0.178
RE3	0.252	0.488	0.307	0.219	0.275	0.488	0.860	0.046	0.092
RE4	0.137	0.376	0.472	0.225	0.305	0.566	0.733	0.076	−0.022
RE5	0.068	0.425	0.278	0.149	0.261	−0.066	0.078	0.969	0.367
RE6	0.049	0.477	0.349	0.247	0.379	0.057	0.161	0.975	0.403
RE7	−0.028	0.211	0.138	0.326	0.254	0.125	0.085	0.306	0.834
RE8	0.052	0.157	0.291	0.289	0.426	0.053	0.189	0.515	0.862
RE9	−0.033	0.047	0.232	0.272	0.389	0.289	0.286	0.178	0.872

, the loads of the items in their respective construct (in bold) is higher than the loads of the items in other constructs [81,98], therefore, confirming their discriminant validity

The final analysis to determine discriminant validity of the HTMT method [99], which is an estimate of what the correlation between constructs would be if they were measured without errors, i.e., perfectly reliable [28].

Table 8. Heterotrait–Monotrait (HTMT) criteria.

	Collaboration Breadth	Organisational Aperture	Coupled	OI Management	IPR Protection	Transference Results	Inbound	Outbound	Scientific Results	Collaboration Depth
Organisational aperture	0.521
Coupled	0.278	0.390
OI Management	0.453	0.303	0.266
IPR Protection	0.409	0.512	0.618	0.395
Transference results	0.324	0.068	0.172	0.494	0.371
Inbound	0.460	0.293	0.544	0.236	0.414	0.271
Outbound	0.385	0.214	0.683	0.491	0.814	0.430	0.640
Scientific results	0.308	0.262	0.689	0.451	0.560	0.149	0.349	0.411
Collaboration depth	0.618	0.528	0.558	0.354	0.429	0.231	0.645	0.662	0.060
Economic results	0.479	0.123	0.279	0.207	0.318	0.442	0.480	0.537	0.292	0.460

shows the HTMT values, which, in all cases, show values lower than 0.85 [80] and, therefore, show discriminant validity.

5.2.4. Second-Order Construct Evaluation

The validation of the second order constructs: performance and OI practices, follows the approach proposed by Duarte and Amaro [96]: (1) evaluate the absence of collinearity, (2) assess nomological validity, and (3) assess discriminant validity. Notice that first level variables that form the second order constructs (inbound, outbound, and coupled for OI practices; scientific, transference, and economic for RTOs performance) have already been validated in the previous step.

To evaluate the absence of collinearity between the first-degree factors and the second-degree construct, the variance inflation factor (VIF) of the first level reflective variables that make up the second level constructs was checked, as shown in

Table 9. Variance inflation factor (VIF) of second order constructs.

	OI Practices	Performance
Coupled	1.653
Transference results		1.224
Inbound	1.333
Outbound	2.221
Scientific results		1.801
Economic results		1.459

VIF values are low, much lower than the value of 5 recommended by Hair et al. [28] and even better than the 3.3 proposed by Diamantopoulos and Siguaw [75]; thus, collinearity does not seem to present problems in the subsequent analysis.

The evaluation of the nomological validity of the construct is made by evaluating the values and significance of the relationship coefficients between the first level variables (reflective) and the second level variables (formative). Using the bootstrapping method [28], the significance values were evaluated and are shown in

Table 10. Relationships among variables in second order constructs.

Second-Order Constructs	First-Level Constructs	Original Sample (O)	Statistics t (|O/STDEV|)	2.5%	97.5%	p Values
OI Practices	Coupled -> OI Practices	0.562	5.563	0.375	0.781	0.000
	Inbound -> OI Practices	0.306	3.616	0.115	0.446	0.000
	Outbound -> OI Practices	0.358	4.496	0.201	0.532	0.000
Performance	Transference results -> Performance	0.364	2.655	0.021	0.510	0.008
	Scientific results -> Performance	0.530	2.573	0.141	0.912	0.010
	Economic results -> Performance	0.517	3.741	0.112	0.657	0.000

As can be seen in the table, the ratio coefficients of the first level variables and the respective constructs range from 0.306 to 0.562, exceeding the level of 0.1 [100], and the p levels, less than 0.05 [28], are significant, which shows the nomological validity of the construct.

In order to evaluate the discriminatory validity, it is verified that the correlation between multi-level constructs and other constructs is less than 0.7 [98].

Table 11. Correlation among structural model variables.

	Collaboration Breadth	Organisational Aperture	OI Management	IPR Protection	Collaboration Depth	OI Practices	Performance
Collaboration breadth	1.000
Organisational aperture	0.515	1.000
OI Management	0.441	0.296	1.000
IPR Protection	0.378	0.451	0.337	1.000
Collaboration depth	0.618	0.527	0.350	0.399	1.000
OI Practices	0.378	0.310	0.336	0.620	0.645	1.000
Performance	0.499	0.124	0.446	0.492	0.334	0.490	1.000

shows the correlations between the different variables of the model, which in all cases are lower than the value of 0.7, the highest value being 0.645, which confirm the discriminant validity of the second level formative constructs.

This analysis finishes the evaluation of the second order variables and completes the measurement model evaluation. Since the evaluation criteria are met, we conclude that all used constructs provide satisfactory levels of quality.

5.3. Structural Model Evaluation

For the evaluation of the structural model we followed the steps proposed by Hair et al. [101]: (1) evaluate the absence of collinearity, (2) evaluate the significance and relevance of the relationships between variables, (3) evaluate the coefficient of determination R² and the effect size f², and (4) evaluate the predictive relevance Q² and the effect size q².

The assessment of collinearity is carried out using the value of the model’s VIF, and the results shown in

Table 12. VIF values of structural model variables.

	Collaboration Breadth	Collaboration Depth	OI Practices	Performance
Collaboration breadth				1.621
Organisational aperture	1.293	1.293	1.293
OI Management	1.162	1.162	1.162
IPR Protection	1.331	1.331	1.331
Collaboration depth				2.379
OI Practices				1.714

are below the demanding value of 3.3 [75].

The relationship coefficients between variables and their significance are then evaluated.

Table 13. Structural model relationships.

	Original Sample (O)	Sample Mean (M)	Standard Deviation (STDEV)	Statistics t (|O/STDEV|)	2.5%	97.5%	p Values
Collaboration breadth -> Performance	0.490	0.484	0.154	3.175	0.141	0.745	0.002
Organisational aperture -> Collaboration breadth	0.379	0.390	0.118	3.224	0.148	0.618	0.001
Organisational aperture -> Collaboration depth	0.404	0.416	0.142	2.843	0.132	0.684	0.004
Organisational aperture -> OI Practices	0.013	0.010	0.131	0.097	−0.252	0.258	0.922
OI Management -> Collaboration breadth	0.293	0.300	0.157	1.861	−0.030	0.588	0.063
OI Management -> Collaboration Depth	0.177	0.190	0.166	1.070	−0.139	0.496	0.285
OI Management -> OI Practices	0.142	0.139	0.160	0.888	−0.210	0.424	0.374
IPR Protection -> Collaboration breadth	0.108	0.100	0.182	0.595	−0.261	0.457	0.552
IPR Protection -> Collaboration Depth	0.157	0.142	0.174	0.902	−0.220	0.463	0.367
IPR Protection -> OI Practices	0.567	0.572	0.109	5.193	0.328	0.767	0.000
Collaboration depth -> Performance	−0.284	−0.281	0.222	1.278	−0.699	0.173	0.201
OI Practices -> Performance	0.488	0.473	0.175	2.798	0.128	0.805	0.005

shows the values obtained for the relationship coefficients, with the mean values and their standard deviation, as well as the values of t and their level of significance. Likewise, following the recommendations of several authors [28,81,98] confidence levels are also reported at 2.5% and 95%.

Regarding the organisational commitment towards OI, results show statistically significant relationships: organisational aperture positively affects collaboration breadth and collaboration depth. OI management positively affects collaboration breadth, and IPR positively affects OI practices. On the other hand, regarding the effect of OI application in performance, results show statistically significant relations: both collaboration breadth and OI practices positively affect RTOs’ performance.

Now, let us consider additional analyses to evaluate the structural model (and individual relationships) using the coefficient of determination R², which is the most commonly used measure to evaluate the structural model [28], and the predictive relevance Q² to evaluate the model’s predictive accuracy.

Table 14. Coefficient of determination (R² values).

	R2	Adjusted R2
Collaboration breadth	0.365	0.308
Collaboration depth	0.337	0.277
OI Practices	0.403	0.349
Performance	0.389	0.334

shows the values of the coefficient of determination R², as well as the adjusted R² [101].

Complementing the coefficient of determination R², we evaluated the effect size f² to measure the individual effect of variables in endogenous constructs [81,98] as shown in

Table 15. Coefficient of determination effect size (f²).

	Collaboration Breadth	Collaboration Depth	OI Practices	Performance
Collaboration breadth				0.243
Organisational aperture	0.175	0.191	0.000
OI Management	0.116	0.041	0.029
IPR Protection	0.014	0.028	0.404
Collaboration depth				0.056
OI Practices				0.228

. Following the criteria indicated by Hair et al. [101], it is established that values of 0.01, 0.15, and 0.35 have a small, medium, and large effect, respectively. Both collaboration breadth and OI practices have a medium f² size effect on RTOs’ performance. For its part, the organisational aperture variable has a medium effect on the collaboration breadth and collaboration depth but no effect on OI practices. OI management has a small effect on collaboration breadth and a very small effect on collaboration depth and OI practices. IPR protection has a small effect on collaboration breadth and collaboration depth and a large effect on OI practices.

Additionally, in order to evaluate the predictive capacity (Q²) of the model, the blindfolding procedure [28] is carried out to obtain Q² values. As can be seen in

Table 16. Predictive capacity (Q²) values.

	Q2
Collaboration breadth	0.226
Collaboration depth	0.200
OI Practices	0.336
Performance	0.189

Q² values for all endogenous variables are positive, which shows predictive relevance [74,98].

The values of effect size q² [28,74] that measure the relative importance of the variables in the value of Q² are shown in

Table 17. Predictive capacity effect size (q²).

	Collaboration Breadth	Collaboration Depth	OI Practices	Performance
Collaboration breadth				0.028
Collaboration depth				−0.048
OI Practices				0.032
IPR Protection	−0.051	−0.001	0.117
OI Management	0.017	−0.023	−0.003
Organisational aperture	0.038	0.033	−0.011

. Following the criteria indicated by Hair et al. (2017a), it is established that values of 0.01, 0.15, and 0.35 have a small, medium, and large effect, respectively. The values show that the effect of collaboration breadth and OI practices on performance is small. The effect of the organisational aperture on collaboration breadth and collaboration depth is small. IPR protection has a small effect size on OI practices. The OI management has a small effect size on the collaboration breadth.

5.4. Non-Linear Effects of Collaboration Breadth and Depth in Performance

We also performed additional analysis to evaluate non-linear relations in the model. Due to transaction costs associated with the collaboration with different partners, there may be diminishing effects of collaboration breadth and depth in performance, with an inverted U shape [37,63,64].

To check for this effect in RTOs, we added the quadratic effects of both collaboration breadth and depth in performance to the original PLS-SEM model (Figure 2), using the SmartPLS quadratic effect tool [95]. The new model evaluation (

Table 18. Quadratic effect of collaboration depth and breadth in performance.

	Original Sample (O)	Sample Mean (M)	Standard Deviation (STDEV)	Statistics t (|O/STDEV|)	2.5%	97.5%	p Values
Collaboration breadth -> Performance	0.470	0.454	0.177	2.654	0.077	0.780	0.008
Organisational aperture -> Collaboration breadth	0.379	0.383	0.119	3.183	0.138	0.602	0.001
Organisational aperture -> Collaboration depth	0.404	0.407	0.147	2.748	0.110	0.687	0.006
Organisational aperture -> OI Practices	0.013	0.009	0.128	0.100	−0.246	0.255	0.921
Collaboration breadth squared -> Performance	−0.067	−0.079	0.151	0.443	−0.389	0.196	0.658
Collaboration depth squared -> Performance	0.028	−0.018	0.177	0.157	−0.403	0.293	0.875
OI Management -> Collaboration breadth	0.293	0.297	0.161	1.826	−0.037	0.592	0.068
OI Management -> Collaboration Depth	0.177	0.185	0.167	1.064	−0.144	0.500	0.287
OI Management -> OI Practices	0.142	0.131	0.158	0.899	−0.205	0.415	0.369
IPR Protection -> Collaboration breadth	0.109	0.093	0.180	0.602	−0.254	0.454	0.547
IPR Protection -> Collaboration Depth	0.157	0.146	0.171	0.921	−0.200	0.471	0.357
IPR Protection -> OI Practices	0.567	0.574	0.108	5.250	0.349	0.772	0.000
Collaboration depth -> Performance	−0.235	−0.230	0.265	0.886	−0.737	0.299	0.376
OI Practices -> Performance	0.469	0.467	0.203	2.308	0.055	0.884	0.021

) shows no statistically significant effect of the quadratic values (collaboration breadth squared -> performance and collaboration depth squared -> performance) in performance. The remaining model relations do show similar values in their relation coefficients, maintaining their statistical significance, thus suggesting robust model relationships.

Moreover, the model with quadratic effects does not provide a better model fit since it does not improve the adjusted R² value [28], as shown

Table 19. Coefficient of determination (R² values) with quadratic effects.

	R2	Adjusted R2
Collaboration breadth	0.366	0.308
Collaboration depth	0.338	0.277
OI Practices	0.403	0.349
Performance	0.394	0.296

, compared with the original model (Table 14). These results discard the inverted U relation among collaboration breadth and performance and collaboration depth and performance in our sample.

These analyses complete the structural model evaluation, of which the results will be discussed next, together with the evaluation of the research hypothesis.

6. Discussion and Implications

By answering to the research questions regarding the effect of OI in the performance of the RTOs, as well as the implication of organisational and management issues in the OI application in this type of organisation, this study contributes to the understanding of OI modes and practices in RTOs. The empirical findings obtained using the PLS-SEM structural modelling provides evidence of the impact of OI application in RTOs.

Regarding the collaboration partners, we found different effects. On one side, we found a statistically significant effect of collaboration breadth on the RTOs performance (Table 13), confirming hypothesis H1. This result matches the findings of other authors in different types of organisations: big companies [102], SMEs [7,103] and service companies [47]. However, we found no statistically significant effect of collaboration depth in RTOs’ performance (Table 13), so hypothesis H2 is rejected, which is a partially unexpected result. Several authors have found a positive effect of collaboration breadth on performance [37,40,64]. However, other authors have also found no positive effect collaboration depth on performance in biotechnology firms [104] or the electronic component industry [85]. In the same line, and in the Spanish context, other authors have found no positive effect of collaboration depth in performance [105] in high-tech manufacturing companies involved in technologically stable contexts. These differences in results could be due to the downsides associated with collaboration, that can surpass its benefits [105], and suggest that in a high-tech context, collaboration depth is not so important. Furthermore, no decreasing effect of the number of collaborating partners in RTOs’ performance has been found in both collaboration breadth and depth. Interestingly, it seems that RTOs can benefit from collaborating partners without the need to establish deep connections and that they are able to profit from increasing the number of partners types without suffering diminishing effects related to the costs associated with managing those collaborations. This effect may be related to the R&D nature and high absorptive capacity of RTOs, as some authors have shown that in high technology contexts, like the electronic industry [85], biotechnology [104], or high-tech manufacturing [105], collaboration depth is not related to better performance. Additionally, the open nature of RTOs as an intermediary in innovation systems may allow them to develop the necessary skills to benefit from increasing the number of collaborating partner types.

We also found a positive and significant effect of the OI practices in RTOs’ performance (see Table 13), confirming hypothesis H3, which indicated that RTOs that make greater use of different OI practices in their three main innovation modes (inbound, outbound, and coupled) achieve greater performance. This matches the results of other authors [54] that have found a positive effect of a variety of OI practices in firms.

Additionally, we explored the effect of organisational commitment towards OI, measured by organisational aperture, OI management, and IPR protection in OI application. As it could be expected, organisation aperture, measured as the willingness of the organisation to collaborate, has a positive effect on both the collaboration breadth and depth (Table 13), since the organisation characteristics facilitate the development of collaborative relations with different partners. On the other hand, no effect of organisational aperture on the use of OI practices has been found (see Table 13), possibly since fostering collaborative relationships could be developed using either a high practice variety or concentrating on a small number of practices.

OI management has a positive effect on the collaboration breadth, with a significance of p < 0.1. However (see Table 13), empirical evidence does not show a significant effect of OI management on either the collaboration depth or OI practices.

Finally, IPR protection has a significant effect on the use of OI practices (see Table 13). This could be expected because OI practices demand the control of the knowledge assets (i.e., patents for technology licensing). Additionally, the use of different practices in multiple collaborative relationships requires high control and management of the IPR protection mechanisms.

7. Conclusions, Limitations, and Further Research

Since its conceptualisation in 2003 by Chesbrough [3], OI has been both studied and implemented in many organisations, raising the interest of academia and practitioners. Research and development organisations (RTOs) are paradigmatic OI agents at the heart of innovation systems, and they develop their own R&D activities connecting innovation system actors to foster industrial innovation. Due to their role, they need to develop OI approaches to balance their knowledge stocks and flows, while assuring their own innovation capabilities, positively impacting the innovation system they influence and ensuring its long-term sustainability. OI literature, however, has concentrated in firms, especially big companies, although SMEs and service companies are increasingly being considered. RTOs have been mainly indirectly studied as collaborators and partners of different types of private firms in an OI context, rather than as the beneficiaries of the OI paradigm [24]. This research addresses this gap with the objective to analyse the OI phenomenon in RTOs, evaluate the effect of OI in the overall performance of RTOs, and study how organisational and managements factors affects their OI approaches.

The research considers a sample of Spanish RTOs and raises substantive conclusions on their OI approach and impacts related to their singular nature. First, results show there is a significant effect of the collaboration with different partner types (collaboration breadth) in RTOs overall performance. Moreover, deepening the collaboration (collaboration depth) with those partners does not significantly affect RTOs’ performance. Interestingly, results suggest that the effect of collaboration with different partners do not have a limit in which collaboration costs outweigh the benefits. Probably, the special characteristics of RTOs as an R&D organisation, with a high level of absorptive capacity and its intermediary role in the innovation systems, makes them able to benefit from every external collaborating partner type, even if they do not develop deep relationships.

Second, the use of different OI practices has a strong impact on RTOs’ overall performance. As expected, using different OI practices will increase RTOs’ capacity to acquire and exploit knowledge, accelerating their innovation cycles, and increase the knowledge transfer channel, thus developing a more diversified and sustainable business model.

Third, organisational and managerial factors play a big role in developing an OI strategy in RTOs. Therefore, they need to foster their internal organisational aperture and promote active management to engage with external collaborating partners, whereas they should pay special attention to questions related to the protection and management of intellectual property when promoting the use of different OI practices. Even though it may seem contradictory at first glance to apply IPR protection strategies in an organisation oriented to provide knowledge and technology to firms that receive partial public funding, RTOs also need to capture the value from their knowledge transactions to complement their funding sources and ensure their own long-term sustainability. Thus, promoting the flow of knowledge and, at the same time, protecting their own knowledge assets becomes a crucial challenge for RTOs.

About the academic value of this paper, we argue that this research contributes to the advance of the knowledge of OI in different areas. First, we contributed by extending the OI approach in a new context, i.e., RTOs that are non-profit research organisations with an intermediary role in the innovation systems that have been very scarcely studied from the OI perspective [13,23]. Second, building off of extant literature, we proposed a new PLS-SEM framework to study OI in RTOs, combining simultaneous effect of collaborating partners (using broad and deep collaborations) and different OI practices in performance. The framework proposes and validates new measures to capture the complexity of OI practices in RTOs, including its three different OI modes: inbound, outbound, and coupled. In the same way, new measures have been developed and validated to measure the multidimensional character of RTOs performance, including scientific output, technology transfer, and economic results. Third, it shows the effect of collaborating with external partners in the performance of RTOs that differ from those found in firms [7,8,9,10,37]. Fourth, the research contributes to increase our understanding of the organisational and managerial factors to be considered when developing OI approaches in RTOs.

These findings provide practical contributions for RTOs stakeholders, generating insights into the OI approaches and impact. For RTOs’ managers, the results suggest that adopting a more open approach to innovation can be beneficial. Since RTOs’ performance is positively affected by collaboration breadth and OI practices, managers could create a fertile ground within the RTOs that facilitates the collaboration with external partners and the use of different OI practices. First, they could increase the organisational openness, fostering an open attitude and culture towards external collaborations. Second, they could develop more formal OI management procedures to efficiently direct and manage external collaborations toward the specific objectives. Both aspects could positively help to develop a broader collaboration network. Third, the development of an active management of its own knowledge assets paying attention to their protection practices would be an adequate ground to increase the use of a variety of OI practices.

Public administrations play a big role in RTOs, and many of them were set up within innovation system policy frameworks to deploy their policies to foster industrial innovation. Rapid technological evolution and internationalisations are important driving forces creating a more open and connected innovation ecosystem. An efficient innovation system should be able to rapidly translate scientific development into industrial innovation, thus requiring efficient RTOs to catalyse the transfer of knowledge and technology to industry. In this context, RTOs should evolve to continue being a relevant player, improving their performance and contribution to the dynamisation of the innovation system. Thus, public administration could help RTOs adopt a more connected and open approach towards innovation in order to increase the overall effectiveness of the innovation policies. Specifically, they could positively influence RTOs’ performance with funding policies to foster the development of a broader collaboration with other innovation system actors and promote the use of a variety of OI practices to increase their knowledge and technology transfer channels.

In this study, some limitations arise from the nature of the work. One is related to the use of a self-reported survey with performance measures. Despite the use of similar techniques in many other similar studies [47,106,107], evidence-based variables and measures regarding OI and RTOs’ performance could be a more precise approach. Another limitation is related to the low number of RTOs used in the study, which, although highly representative of the population in Spain, suggests that results should be interpreted with caution when extending them to other geographical and innovation systems’ contexts.

Future research activities could focus on investigating the relation among OI variables (collaboration breadth, collaboration depth, and OI practices) and identifying the most beneficial combination of these variables for RTOs. Additionally, it could be interesting to study the effect of different collaborating partners types in RTOs’ performance, using different measures to complement the effect of collaboration breadth and depth used in this study. Finally, the extension of the research to other countries could also be interesting from an innovation system perspective.