Next Article in Journal
Advancements in AI-Based Information Technologies: Solutions for Quality and Security
Previous Article in Journal
Redesigning the Drugs Distribution Network: The Case of the Italian National Healthcare Service
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Quantity or Quality? The Impact of Multilevel Network Structural Holes on Firm Innovation

School of Management, Shanghai University, Shanghai 200444, China
*
Author to whom correspondence should be addressed.
Systems 2024, 12(2), 57; https://doi.org/10.3390/systems12020057
Submission received: 15 January 2024 / Revised: 7 February 2024 / Accepted: 7 February 2024 / Published: 9 February 2024

Abstract

:
Embedding collaboration networks in the context of open innovation can facilitate firm innovation. Previous studies have not considered the impact of multilevel network structural embedding on firm innovation. In this study, organizational collaboration networks, knowledge networks, and urban collaboration networks are viewed as systems to explore their impact on innovation quantity and innovation quality. We validate the research hypotheses using data from Chinese high-tech firms in the field of artificial intelligence and intelligent manufacturing equipment. The results indicate that structural holes occupied by firms in organizational collaboration networks can increase the innovation quantity and have a U-shaped effect on innovation quality. Knowledge network structural holes and urban collaboration network structural holes moderate the relationship between organizational collaboration network structural holes and innovation quantity and quality. Our findings will help firms to efficiently utilize the advantages of multilevel network structural holes to improve the innovation quantity and innovation quality.

1. Introduction

Innovation activities are becoming increasingly complex, and the resources a firm has are limited and do not meet the needs of technological innovation [1]. Collaborative innovation has become an important strategy for firms to cope with difficult innovation activities and increase innovation output. Organizational collaboration networks consisting of inter-organizational collaborative relationships have received attention from existing studies that agree that embedding collaboration networks helps to promote firm innovation [2,3]. Firms with structural advantages in organizational collaboration networks are more likely to access heterogeneous resources [4]. According to the resource-based view (RBV) [5], valuable and scarce heterogeneous resources can provide a competitive advantage for a firm. The heterogeneous resources brought to the firm by the structural advantages of the network will have an important impact on the firm’s innovation. What kinds of structural advantages a firm maintains in its organizational collaboration network are most conducive to innovation is a key issue that firms need to clarify.
The knowledge-based view (KBV) sees knowledge resources as an important foundation for firms to maintain their innovative advantage [6]. Different types of organizations possess different knowledge resources, and inter-organizational collaborative innovation activities provide opportunities for knowledge recombination [7]. Knowledge networks were first defined by Yayavaram and Ahuja [8], who proposed that a knowledge network is a network of knowledge elements as nodes covering the combinatorial relationships between them. As technological innovation becomes increasingly complex and the knowledge resources possessed by firms are insufficient to support innovation activities, knowledge exchange with other organizations builds external channels for firms to absorb knowledge [2,9]. The combination and utilization of knowledge elements in a knowledge network can have an impact on firm innovation [10].
The city is the spatial carrier of innovation activities, the firm is the implementation of innovation activities, and firm innovation is affected by the urban innovation environment [11]. Collaborative relationships between different cities form urban collaboration networks, which allow innovation resources to flow between cities [12]. Structural features in the urban collaboration networks have an impact on the urban resources, and the accumulation of resources affects the urban innovation environment and, thus, further influences the firm innovation within the city.
The organizational collaboration network highlights the importance of searching for resources based on inter-organizational partnerships [13], the knowledge network highlights the importance of searching for resources based on knowledge elements [2,10], and the urban collaboration network highlights the importance of searching for resources based on inter-city partnerships [12]. Most of the existing studies focus on the impact of organizational collaboration networks on firm innovation [14], and some studies focus on the impact of collaboration networks and knowledge networks on firm innovation [2,10], but most of them revolve around innovation performance and dual innovation (exploitative and exploratory innovations) and fail to explore the quantity and quality of firm innovation. Guan et al. [15] pointed out that different levels of networks are interconnected with each other. Therefore, when studying firm innovation from a network perspective, we need to combine the characteristics of different levels of networks and consider the impact of the interaction between different levels of networks on firm innovation.

2. Theory Analysis and Hypotheses Development

2.1. Organizational Collaboration Network Structural Holes and Firm Innovation

Organizational collaboration networks consisting of complex inter-organizational collaborative relationships bring together heterogeneous resources and form a stable network structure, which increases the potential opportunities for firm innovation [3]. The research related to collaboration networks and innovation began with the definition of Freeman [16], who emphasized that collaboration networks are network innovation models based on formal or informal relationships between organizations to access external resources to reduce risk. From the perspective of social network theory, the existence of structural holes in an organizational collaboration network indicates that the focal firm occupies a “bridge” position in the network. Firms that span structural holes are non-redundantly connected in the network [17]. According to Burt’s structural hole theory [18], non-redundant ties can provide firms with “information benefits” and “control benefits”. Structural holes place the firm in a position between two actors that are not interconnected, acting as a bridge to other actors in the network [18]. Firms occupying structural holes have greater access to diverse resources, which can have an impact on firm innovation.
Firms occupying structural holes in the network act as mediators of information exchange and resource transfer between different organizations, providing indirect channels of cooperation for organizations that are not directly connected, greatly increasing the possibility of heterogeneous knowledge and resources being shared and overflowed within the network [19]. Firms that span structural holes connect organizations that are not connected to each other and are more likely to absorb novel technological knowledge. The absorption and utilization of novel knowledge can broaden the technological innovation thinking of firms and improve the innovativeness of new products, thus increasing the innovation output of firms [20]. Compared with firms in the same technological field, firms occupying structural holes can prioritize access to external heterogeneous resources and develop a control advantage, which increases the innovation speed of firms and earns them a first-mover advantage in terms of innovation quantity. Accordingly, we propose Hypothesis 1.
Hypothesis 1 (H1). 
The structural hole that firms occupy in the organizational collaboration network positively affects the innovation quantity.
Unlike the innovation quantity, the innovation quality is more demanding in terms of knowledge utilization and resource allocation [21]. The expansion of knowledge resources has a limited role to play in improving the innovation quality. Although structural holes provide firms with access to resources, they also have a negative impact on firm innovation. When the structural holes are few, there is a large number of inter-organizational ties within the collaboration network, and dense inter-organizational ties provide a foundation of trust for collaboration innovation [22]. Different organizations have a higher willingness to share knowledge in a trustworthy collaboration innovation environment, which promotes the transfer of tacit knowledge in the network and facilitates the absorption of deep knowledge resources by firms to improve innovation quality [23]. With the increase in structural holes, the inter-organizational ties within the network gradually become fewer, which leads to the difficulty of firms accessing the deep knowledge resources of other organizations, which is not conducive to the improvement of innovation quality [24]. When a firm occupies enough structural holes, it can control the direction of information and resource flows within the network and harvest more network capital. Occupying more structural holes puts the firm in an advantageous position in terms of information asymmetry, and the firm can integrate heterogeneous resources from different organizations. Firms with sufficient resources can sustain their innovation activities to improve the innovation quality [25]. Based on this, the following hypothesis is proposed:
Hypothesis 2 (H2). 
The structural hole that firms occupy in the organizational collaboration network has a U-shaped effect on innovation quality.

2.2. The Moderating Effect of Knowledge Network Structural Holes

The knowledge-based view (KBV) recognizes that knowledge is a core resource for firms to gain a competitive advantage [26]. Knowledge networks record the combinatorial relationships of knowledge elements and provide opportunities for the migration and recombination of knowledge elements [8]. Previous studies have demonstrated that collaboration networks and knowledge networks are decoupled [2,10]. Knowledge networks influence firm innovation through a unique mechanism based on knowledge search. Structural holes in knowledge networks reflect the non-redundancy of combinatorial relationships between knowledge elements [27]. The fact that the knowledge elements owned by the firm occupy structural holes in the knowledge network indicates that the firm has not yet achieved a full understanding of the knowledge and has not yet applied such knowledge in depth [28]. Although firms occupying structural holes in organizational collaboration networks have access to more heterogeneous resources, firms do not possess the technological knowledge to exploit these resources, making it difficult to increase the innovation output in the short term and, thus, difficult to increase the innovation quantity [29].
Knowledge elements occupying structural holes in knowledge networks link knowledge from different technical fields, and firms can explore more novel knowledge combinations based on the knowledge elements occupying structural holes [30]. With the help of knowledge elements occupying structural holes in knowledge networks, firms can assimilate novel technological knowledge that is distant from the existing knowledge and avoid falling into a technological lock-in [31]. When efficiently utilizing the heterogeneous resources brought about by the structural holes in the organizational collaboration network, firms can use novel technological knowledge to stimulate innovation, break through the constraints of the old innovation paths, and carry out high-quality innovation activities in a more efficient way to improve the innovation quality. Therefore, we propose the following hypotheses.
Hypothesis 3a (H3a). 
Structural holes in knowledge networks negatively moderate the relationship between structural holes in organizational collaboration networks and the innovation quantity.
Hypothesis 3b (H3b). 
Structural holes in knowledge networks positively moderate the relationship between structural holes in organizational collaboration networks and the innovation quality.

2.3. The Moderating Effect of Urban Collaboration Network Structural Holes

Cities are spatial carriers of firm innovation, and they provide a favorable innovation environment for collaborative innovation among different organizations [32]. Firm innovation is embedded in the city, and firms and cities are an integrated system. Burt [18] demonstrated that structural holes provide opportunities to enable knowledge and information flow among network members. Cities that occupy structural holes in urban collaboration networks have an informational advantage and more autonomy in their innovation activities. However, structural holes can also have certain negative effects. Decentralized cities in an urban collaboration network have difficulty coordinating all types of innovation resources in a sparse network [33]. Structural holes in urban collaboration networks reduce mutual trust between cities [34], which can lead to firms not being able to access urban resources quickly, which has a limited effect on the promotion of innovation output and makes it difficult to increase the innovation quantity.
Unlike the innovation quantity, improvement in the innovation quality requires a higher level of resource support, and investing large amounts of innovation resources in the short term does not immediately improve the innovation quality. In the long run, efficiently allocating and utilizing heterogeneous resources is more helpful for the innovation quality. The resource-based view (RBV) assumes that actors with more heterogeneous resources can perform better in innovation [35]. Cities occupying structural holes in urban collaboration networks are located at the intersection of different domains and have greater access to non-redundant and heterogeneous resources that provide a competitive advantage to firm innovation within the city [36]. Structural holes in the urban collaboration network bring city-level resource support for firm innovation, which enhances the organizational-level resource advantages provided by structural holes in the organizational collaboration network and promotes innovation quality. We, therefore, propose the following hypotheses:
Hypothesis 4a (H4a). 
Structural holes in urban collaboration networks negatively moderate the relationship between structural holes in organizational collaboration networks and the innovation quantity.
Hypothesis 4b (H4b). 
Structural holes in urban collaboration networks positively moderate the relationship between structural holes in organizational collaboration networks and the innovation quality.

2.4. Interaction between Structural Holes in Multilevel Networks

When analyzing firm innovation from a network perspective, we need to recognize that structural holes in networks at different levels have different effects on firm innovation. Different levels of networks are decoupled from each other, and different levels of networks interact in influencing firm innovation [15,37]. We consider the organizational collaboration network, the knowledge network, and the urban collaboration network as a system, and the interaction between the three network levels is shown in Figure 1.
Firm innovation requires a continuous investment of knowledge resources [38]. A knowledge network records the combined relationships between many knowledge elements. Knowledge elements spanning structural holes linking knowledge elements from technological fields that are not yet interconnected can help firms explore new paradigms of knowledge combinations [39]. Firm innovation requires the support of urban resources, and urban collaboration networks constituted by partnerships between different cities influence the access of cities to innovation resources [40]. Cities that span structural holes have a higher priority in terms of access to resources, providing more opportunities to acquire external resources for firm innovation within the city [36]. The knowledge advantage provided by the structural holes in the knowledge network can help firms efficiently allocate the urban resources provided by the structural holes in the urban collaboration network and organizational resources provided by the structural holes in the organizational collaboration network. The information benefits of structural holes in three-level networks can help firms avoid the negative effects that can result from structural holes in a single-level network [41].
Firms with knowledge elements that occupy structural holes in the knowledge network have access to differentiated knowledge resources [10]. Firms occupying structural holes in organizational collaboration networks have access to heterogeneous resources at the organizational level [19,42]. Cities where firms are located across more structural holes can capture a larger range of innovation resources from urban collaboration networks [36,43]. The structural holes in the different levels of the network bring complementary resources to firms, providing both sustained resource support to increase the innovation quantity and cross-disciplinary, cutting-edge information and knowledge to improve the innovation quality. Therefore, we posit the following hypotheses:
Hypothesis 5a (H5a). 
The impact of the structural holes occupied by firms in organizational collaboration networks on the innovation quantity is strengthened when the city in which the firm is located occupies a structural hole in the urban collaboration network and the firm possesses a knowledge element that occupies a structural hole in the knowledge network.
Hypothesis 5b (H5b). 
The impact of the structural holes occupied by firms in organizational collaboration networks on the innovation quality is strengthened when the city in which the firm is located occupies a structural hole in the urban collaboration network and the firm possesses a knowledge element that occupies a structural hole in the knowledge network.
Based on the above analysis, the theoretical framework is presented in Figure 2.

3. Research Design

3.1. Sample Selection and Data Source

High-tech firms in China are actively engaged in innovative activities. We selected Chinese high-tech enterprises in the field of artificial intelligence and intelligent manufacturing equipment as the research sample. Previous studies have pointed out that jointly authorized invention patents can effectively indicate collaborative innovation relationships between organizations [2,13]. Therefore, this study used jointly authorized invention patent data to construct a multilevel network. We downloaded jointly authorized invention patents in the field of artificial intelligence and intelligent manufacturing equipment in China from the PATSNAP (www.zhihuiya.com accessed on 15 September 2022). We set the number of patent owners to be greater than or equal to 2 and excluded patent data where the patent owner was an individual. We set the patent application period to 2011–2020, and we obtained 22,576 patents.
Drawing on existing research [3,14], we constructed organizational collaboration networks based on patentee information in the jointly authorized invention patent data. We utilized TianYanCha (www.tianyancha.com accessed on 15 September 2022) to query the geographic locations of patentees and build urban collaboration networks based on the geographic information of the patentees [44]. Previous studies have pointed out that IPC codes can be used as a categorization criterion for knowledge elements [2,10]. We constructed a knowledge network based on the co-occurrence information of IPC codes in patents. Consistent with previous studies [19,41], we set the time window for collaboration networks and knowledge networks to 3 years. Therefore, we divided 2011–2020 into 8 time windows and collected unbalanced panel data for a total of 6331 sample points from 2011 to 2020. The patent data in this study came from PATSNAP, and the firm’s business information came from the CSMAR database, annual reports, and TianYanCha.

3.2. Measurement

3.2.1. Dependent Variable

Innovation quantity: A direct reflection of the innovation quantity is the number of patent applications, including invention patents, utility model patents, and industrial design patents. Existing studies on the measurement of the innovation quantity use the number of patent applications of a firm [45]. Consistent with previous studies [45], we used the number of patent applications to measure the innovation quantity. Firms will innovate in cooperation with other organizations immediately after embedding in organizational collaboration networks but will not apply for patents immediately. There is a lag effect in the impact of organizational collaboration networks on firm innovation. Therefore, we selected the innovation quantity of firms embedded in the organizational collaboration network lagging by one year as the explanatory variable.
Innovation quality: Innovation quality was first defined by Haner [46], who argued that it emphasizes the impact of a firm’s innovations. Existing studies have confirmed the important role of patents in measuring innovation quality [47]. Drawing on existing studies [48], we used the number of citations of patents to measure innovation quality. Similarly, considering the lag effect, we selected the innovation quality of firms embedded in organizational collaboration networks that lagged by one year as an explanatory variable.

3.2.2. Independent Variable

Structural holes in organizational collaboration networks (SHO): The constraint indicator for structural holes indicates the degree of connectivity around the nodes. Nodes with high constraints are more connected to each other and nodes with low constraints are less connected to each other. Drawing on Wang et al. [10], we calculated a structural hole by subtracting the constraint from 2. The specific calculation formula is as follows:
C O i = j p i j + q , q i , q j p i q p q j 2 S t h i = 2 C O i
where Sth denotes the structural hole, CO denotes the node’s constraint, and pij is the proportion of node i’s relational inputs to link j.

3.2.3. Moderator Variables

Structural holes in knowledge networks (SHK): We also used Equation (1) to compute the structural holes in the knowledge network. Firms usually hold multiple knowledge elements, and we needed to aggregate the characteristics of the knowledge elements at the firm level. Drawing on Wang et al. [10], we utilized arithmetic averaging to aggregate the knowledge elements. For example, if a firm has 3 knowledge elements, and the structural holes of the 3 knowledge elements are Z 1 , Z 2 , Z 3 , then the knowledge structural holes of the firm after aggregation at the firm level are ( Z 1 + Z 2 + Z 3 ) / 3 .
Structural holes in urban collaboration networks (SHUs): We also used Equation (1) to calculate the value of structural holes in the city where the firm was located.

3.2.4. Control Variables

We controlled for other variables that affect firm innovation. Scale of the firm (SCA): We used the natural logarithm of the number of employees to measure the SCA. Firm age (AGE): We measured the AGE using the difference between the year of the observation period minus the year of inception. Knowledge stock (KS): We denoted KS via the natural logarithm of the number of patents granted in the three years before the firm’s embedding in the organizational collaboration network. Degree of centrality (DC): We controlled the firm’s degree of centrality in the organizational collaboration network. We also controlled for the density of organizational collaboration networks (DO), the density of knowledge networks (DK), and the density of urban collaboration networks (DU). The effect of the year was controlled.

3.3. Methods

3.3.1. Social Network Analysis

Social network analysis allows for the quantification of nodes as well as relationships in multilevel networks [3]. The networks constructed in this study included organizational collaboration networks, knowledge networks, and urban collaboration networks. We distinguished the types of organizations based on the patentee information in jointly authorized invention patents. We used UCINET to calculate the relevant network metrics. The organizational collaboration network for 2018–2020 is shown in Figure 3, the knowledge network is shown in Figure 4, and the urban collaboration network is shown in Figure 5.

3.3.2. Negative Binomial Regression

Both innovation quantity and innovation quality are non-negative integers. Because the variance in innovation quantity and quality differed from the mean, and the explanatory variables were over-dispersed, we used negative binomial regression to analyze the data. The negative binomial regression method can effectively avoid bias in regression analysis [49]. We used STATA to analyze the research data.

4. Analysis and Results

The descriptive statistics and correlation coefficients for the variables are shown in Table 1. The difference between the mean and variance in the study data was large, and the data distribution was too discrete, so we used negative binomial regression. Table 1 shows that the correlation coefficients of the variables are not high overall; they are all below 0.7, which indicates that the regression model does not have the problem of multicollinearity.
The negative binomial regression results of the hypotheses tests are shown in Table 2. From Model 1, it can be found that the effect of SHO on the innovation quantity is positive and significant ( β = 0.11 ; p < 0.01 ), verifying Hypothesis 1, which means that firms occupying more structural holes in the organizational collaboration network are conducive to increasing the innovation quantity.
The regression results of Model 5 show that the coefficient of the squared term of SHO is significantly positive ( β = 0.057 ; p < 0.01 ), indicating that the effect of SHO on the innovation quality is U-shaped, which supports Hypothesis 2. The structural holes occupied by firms in organizational collaboration networks have an inhibitory and then facilitating effect on the innovation quality.
Model 2 was used to test the moderating effect of SHK on the relationship between SHO and the innovation quantity. The interaction term between SHK and SHO is significantly negative ( β = 0.079 ; p < 0.01 ), indicating that SHK negatively moderates the relationship between SHO and the innovation quantity, and Hypothesis H3a is supported. Figure 6 presents the negative moderating effect of SHK on the relationship between SHO and the innovation quantity.
Based on Model 6, the interaction term between SHK and SHO2 is positive and significant ( β = 0.07 ; p < 0.05 ), indicating that SHK plays a positive moderating role in the relationship between SHO and the innovation quality, and Hypotheses H3b is supported, the moderating effect of which is plotted in Figure 7.
Model 3 tested the moderating role of SHU in the relationship between SHO and the innovation quantity. The interaction term between SHU and SHO is significantly negative ( β = 0.029 ; p < 0.1 ), supporting Hypothesis H4a. SHU negatively moderates the relationship between SHO and the innovation quantity, as shown in Figure 8.
Based on Model 7, the interaction term between SHU and SHO2 is positive and significant ( β = 0.054 ; p < 0.01), indicating that SHU positively moderates the relationship between SHO and the innovation quality, supporting Hypothesis H4b, the moderating effect of which is plotted in Figure 9.
Model 4 shows that the three-dimensional interaction term between SHO, SHK, and SHU is positive but not significant ( β = 0.014 ; p > 0.1), indicating that SHK and SHU do not play a significant joint moderating effect in the relationship between SHO and the innovation quantity, and Hypothesis H5a is not supported.
Model 8 shows that the three-dimensional interaction terms of SHO2, SHK, and SHU are significantly positive ( β = 0.04 ; p < 0.1), indicating that SHK and SHU play a positive joint moderating effect in the relationship between SHO and the innovation quality, which supports Hypothesis H5b. The joint moderating effect of SHK and SHU is shown in Figure 10.
Robustness Tests
We used the following methods to conduct robustness tests in this study. First, we changed the measurement of the innovation quantity and innovation quality. Compared with utility model patents and industrial design patents, invention patents require more innovation from firms. Drawing on Hu et al. [50], we used the number of invention patents applied to measure the innovation quantity. Drawing on Liu et al. [51], we used the number of citations of invention patents to measure the innovation quality. Second, we changed the control variables. We changed DC to betweenness centrality (BC), DO to the average distance in organizational collaboration networks (ADO), DK to the average distance in knowledge networks (ADK), and DU to the average distance in urban collaboration networks (ADU). We measured KS by the number of patents granted in the first 5 years of a firm’s embedded organizational collaboration network. Finally, we changed the regression model. We replaced fixed-effects negative binomial regression with random-effects negative binomial regression. The regression results of the robustness test are shown in Table 3. The regression results of the robustness test are consistent with the previous section. Therefore, this study is reliable.

5. Conclusions and Discussion

We constructed organizational collaboration networks, knowledge networks, and urban collaboration networks using jointly authorized invention patents in the field of artificial intelligence and intelligent manufacturing equipment in China and clarified the impact of multilevel network structural holes on firm innovation. The empirical results of this study demonstrate that firms occupying more structural holes in the organizational collaboration network are beneficial in increasing the innovation quantity but inhibit and then promote the innovation quality. Firms occupying more structural holes in the knowledge network weaken the effect of the structural holes in the organizational collaboration network on the innovation quantity but strengthen the effect of the structural holes in the organizational collaboration network on the innovation quality. If the city where the firm is located occupies a structural hole in the urban collaboration network, the effect of the structural hole in the organizational collaboration network on the innovation quantity is weakened and the effect on the innovation quality is strengthened. The structural holes occupied by the firm in the knowledge network and the structural holes occupied by the city in which the firm is located in the urban collaboration network enhance the impact of the structural holes in the organizational collaboration network on the innovation quality.

5.1. Theoretical Implications

Previous studies have focused on the impact of collaboration networks on innovation performance and dual innovation (exploitative and exploratory innovations) [2,9,19]. Unlike the previous literature [1,14], we contribute to the literature on the relationship between structural embeddedness and firm innovation by exploring the differential impact of the structural holes a firm occupies in a network on the innovation quantity and innovation quality. Firm innovation is embedded in multilevel networks, and different networks exert different impacts [10,15]. We explored the impact of multilevel network structural holes on firm innovation by simultaneously including organizational collaboration networks, knowledge networks, and urban collaboration networks in our analytical framework. This complements related research on multilevel network embeddedness and provides a new perspective to the literature on network embeddedness and firm innovation. Our findings demonstrate that knowledge network structural holes and urban collaboration network structural holes play different moderating effects in the relationship between organizational collaboration network structural holes and firm innovation, which expands the contingency context of the collaboration network and firm innovation. Our findings also advance the related research on the relationship between multilevel network structural features and firm innovation.

5.2. Practical Implications

First, firms should develop appropriate innovation plans based on their innovation goals. Firms that plan to increase their innovation quantity in the short term should try to occupy more bridging positions in their organizational collaboration networks. In the long run, firms that are committed to improving innovation quality can both maintain strong ties in their networks and occupy more bridging positions. Second, firms need to choose appropriate knowledge network layout strategies based on their innovation goals. When firms want to increase their innovation quantity, they should reduce the application of knowledge elements that occupy structural holes. Finally, firms need to rationalize the use of urban resources according to their innovation goals. The city where the firm is located occupying a structural hole in the urban collaboration network can help the firm access more resources and thus improve the innovation quality.

5.3. Limitations and Future Research

Certain limitations of this study can be addressed in future research. First, our research sample was restricted to include only high-tech firms in the field of artificial intelligence and intelligent manufacturing equipment in China, and it remains to be tested whether the findings can be applied to other fields. Second, we measured the innovation quantity and innovation quality based only on patent data, which may be incomplete. Future research could analyze firm innovation from other perspectives such as new product development and innovation transformation. Finally, this study only considered the structural holes in multilevel networks, and future research on the impact of multilevel networks on firm innovation can be developed from other perspectives. Moreover, our analysis of multilevel networks and structural holes only yielded conclusions about patent collaboration, and the identification of knowledge elements was limited to IPC codes. Future research could utilize other data to validate these findings.

Author Contributions

Conceptualization, Y.Z. and J.L.; methodology, J.L. and Q.L.; software, J.L. and Q.L.; validation, Y.Z.; formal analysis, J.L.; investigation, Y.Z.; writing—original draft preparation, J.L. and Q.L.; writing—review and editing, Y.Z.; supervision, Y.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported in part by the National Natural Science Foundation of China (grant no. 71673179).

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors upon request.

Acknowledgments

The authors thank the editors and anonymous referees for their valuable comments.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Xie, X.; Liu, X.; Chen, J. A meta-analysis of the relationship between collaborative innovation and innovation performance: The role of formal and informal institutions. Technovation 2023, 124, 102740. [Google Scholar] [CrossRef]
  2. Guan, J.; Liu, N. Exploitative and exploratory innovations in knowledge network and collaboration network: A patent analysis in the technological field of nano-energy. Res. Policy 2016, 45, 97–112. [Google Scholar] [CrossRef]
  3. Tseng, C.; Lin, S.; Pai, D.; Tung, C. The relationship between innovation network and innovation capability: A social network perspective. Technol. Anal. Strateg. Manag. 2016, 28, 1029–1040. [Google Scholar] [CrossRef]
  4. Llopis, O.; D’Este, P.; Díaz-Faes, A. Connecting others: Does a tertius iungens orientation shape the relationship between research networks and innovation? Res. Policy 2021, 50, 104175. [Google Scholar] [CrossRef]
  5. Burt, R.; Soda, G. Network capabilities: Brokerage as a bridge between network theory and the resource-based view of the firm. J. Manag. 2021, 47, 1698–1719. [Google Scholar] [CrossRef]
  6. Zhang, W.; Jiang, Y.; Zhang, W. Capabilities for collaborative innovation of technological alliance: A knowledge-based view. IEEE Trans. Eng. Manag. 2021, 68, 1734–1744. [Google Scholar] [CrossRef]
  7. Roper, S.; Hewitt-Dundas, N. Knowledge stocks, knowledge flows and innovation: Evidence from matched patents and innovation panel data. Res. Policy 2015, 44, 1327–1340. [Google Scholar] [CrossRef]
  8. Yayavaram, S.; Ahuja, G. Decomposability in knowledge structures and its impact on the usefulness of inventions and knowledge-base malleability. Adm. Sci. Q. 2008, 53, 333–362. [Google Scholar] [CrossRef]
  9. Lyu, T.; Geng, Q.; Zhao, Q. Understanding the efforts of cross-border search and knowledge co-creation on manufacturing enterprises’ service innovation performance. Systems 2023, 11, 4. [Google Scholar] [CrossRef]
  10. Wang, C.; Rodan, S.; Fruin, M.; Xu, X. Knowledge networks, collaboration networks, and exploratory innovation. Acad. Manag. J. 2014, 57, 484–514. [Google Scholar] [CrossRef]
  11. Johnston, A.; Huggins, R. The spatio-relational nature of urban innovation systems: Universities, knowledge intensive business service firms, and collaborative networks. J. Urban Technol. 2016, 23, 29–52. [Google Scholar] [CrossRef]
  12. Feng, Z.; Cai, H.; Chen, Z.; Zhou, W. Influence of an interurban innovation network on the innovation capacity of China: A multiplex network perspective. Technol. Forecast. Soc. Change 2022, 180, 121651. [Google Scholar] [CrossRef]
  13. Shi, X.; Zhang, Q. Network inertia and inbound open innovation: Is there a bidirectional relationship? Scientometrics 2020, 122, 791–815. [Google Scholar] [CrossRef]
  14. Han, S.; Lyu, Y.; Ji, R.; Zhu, Y.; Su, J.; Bao, L. Open innovation, network embeddedness and incremental innovation capability. Manag. Decis. 2020, 58, 2655–2680. [Google Scholar] [CrossRef]
  15. Guan, J.; Zhang, J.; Yan, Y. The impact of multilevel networks on innovation. Res. Policy 2015, 44, 545–559. [Google Scholar] [CrossRef]
  16. Freeman, C. Networks of innovators: A synthesis of research issues. Res. Policy 1991, 20, 499–514. [Google Scholar] [CrossRef]
  17. Ter Wal, A.; Alexy, O.; Block, J.; Sandner, P. The best of both worlds: The benefits of open-specialized and closed-diverse syndication networks for new ventures’ success. Adm. Sci. Q. 2016, 61, 393–432. [Google Scholar] [CrossRef]
  18. Burt, R. Structural holes and good ideas. Am. J. Sociol. 2004, 110, 349–399. [Google Scholar] [CrossRef]
  19. Zang, J. Structural holes, exploratory innovation and exploitative innovation. Manag. Decis. 2018, 56, 1682–1695. [Google Scholar] [CrossRef]
  20. Zhang, Y.; Gao, C.; Wang, J. Financing constraints and innovation performance: The moderating role of the network location of cross-border innovation cooperation among Internet enterprises. Eur. J. Innov. Manag. 2023, 26, 1473–1499. [Google Scholar] [CrossRef]
  21. Duan, Y.; Yang, M.; Huang, L.; Chin, T.; Fiano, F.; de Nuccio, E.; Zhou, L. Unveiling the impacts of explicit vs. tacit knowledge hiding on innovation quality: The moderating role of knowledge flow within a firm. J. Bus. Res. 2022, 139, 1489–1500. [Google Scholar] [CrossRef]
  22. An, X.; Deng, H.; Chao, L.; Bai, W. Knowledge management in supporting collaborative innovation community capacity building. J. Knowl. Manag. 2014, 18, 574–590. [Google Scholar] [CrossRef]
  23. Ganguly, A.; Talukdar, A.; Chatterjee, D. Evaluating the role of social capital, tacit knowledge sharing, knowledge quality and reciprocity in determining innovation capability of an organization. J. Knowl. Manag. 2019, 23, 1105–1135. [Google Scholar] [CrossRef]
  24. Ahuja, G. Collaboration networks, structural holes, and innovation: A longitudinal study. Adm. Sci. Q. 2000, 45, 425–455. [Google Scholar] [CrossRef]
  25. Andersén, J. A relational natural-resource-based view on product innovation: The influence of green product innovation and green suppliers on differentiation advantage in small manufacturing firms. Technovation 2021, 104, 102254. [Google Scholar] [CrossRef]
  26. Zhou, K.; Li, C. How knowledge affects radical innovation: Knowledge base, market knowledge acquisition, and internal knowledge sharing. Strateg. Manag. J. 2012, 33, 1090–1102. [Google Scholar] [CrossRef]
  27. Liao, Y.; Phan, P. Internal capabilities, external structural holes network positions, and knowledge creation. J. Technol. Transf. 2016, 41, 1148–1167. [Google Scholar] [CrossRef]
  28. Xia, C.; Li, C. Art of saying no: Linking trust structural hole to knowledge hiding and creativity. Asia Pac. J. Manag. 2023. [Google Scholar] [CrossRef]
  29. Wen, J.; Qualls, W.; Zeng, D. To explore or exploit: The influence of inter-firm R&D network diversity and structural holes on innovation outcomes. Technovation 2021, 100, 102178. [Google Scholar] [CrossRef]
  30. Tang, C.; Liu, L.; Xiao, X. How do firms’ knowledge base and industrial knowledge networks co-affect firm innovation? IEEE Trans. Eng. Manag. 2023, 70, 29–39. [Google Scholar] [CrossRef]
  31. Zhao, L.; Xiang, Y.; Yi, Q. Fuzzy front end patent management and innovation performance Mediating role of patent commercialization and moderating effect of technological lock-in. Manag. Decis. 2017, 55, 1143–1162. [Google Scholar] [CrossRef]
  32. Zhao, Y.; Lyu, J.; Huesig, S. The impact of innovative city cooperation network on city’s innovation efficiency: Evidence from China. J. Knowl. Econ. 2023. [Google Scholar] [CrossRef]
  33. Song, Y.; Kim, J. The spatial spillover effect of technological innovation network in cities: A case of the high-tech industry of Yangtze River Delta. Int. J. Urban Sci. 2023, 27, 414–441. [Google Scholar] [CrossRef]
  34. Ma, D.; Zhang, Y.; Zhang, F. The influence of network positions on exploratory innovation: An empirical evidence from China’s patent analysis. Sci. Technol. Soc. 2020, 25, 184–207. [Google Scholar] [CrossRef]
  35. Arya, B.; Lin, Z. Understanding collaboration outcomes from an extended resource-based view perspective: The roles of organizational characteristics, partner attributes, and network structures. J. Manag. 2007, 33, 697–723. [Google Scholar] [CrossRef]
  36. Yao, L.; Li, J.; Li, J. Urban innovation and intercity patent collaboration: A network analysis of China’s national innovation system. Technol. Forecast. Soc. Change 2020, 160, 120185. [Google Scholar] [CrossRef]
  37. Xie, H.; Zou, J.; Mu, L. Multiple networks and enterprise innovation based on the perspective of middle managers. Knowl. Manag. Res. Pract. 2021, 19, 387–395. [Google Scholar] [CrossRef]
  38. Ode, E.; Ayavoo, R. The mediating role of knowledge application in the relationship between knowledge management practices and firm innovation. J. Innov. Knowl. 2020, 5, 209–217. [Google Scholar] [CrossRef]
  39. Yoon, S.; Marhold, K.; Kang, J. Linking the firm’s knowledge network and subsequent exploratory innovation: A study based on semiconductor industry patent data. Innov.-Organ. Manag. 2017, 19, 463–482. [Google Scholar] [CrossRef]
  40. Yang, W.; Fan, F.; Wang, X.; Yu, H. Knowledge innovation network externalities in the Guangdong-Hong Kong-Macao Greater Bay Area: Borrowing size or agglomeration shadow? Technol. Anal. Strateg. Manag. 2022, 34, 1020–1037. [Google Scholar] [CrossRef]
  41. Zhang, Z.; Luo, T. Knowledge structure, network structure, exploitative and exploratory innovations. Technol. Anal. Strateg. Manag. 2020, 32, 666–682. [Google Scholar] [CrossRef]
  42. Tan, J.; Zhang, H.; Wang, L. Network closure or structural hole? The conditioning effects of network-level social capital on innovation performance. Entrep. Theory Pract. 2015, 39, 1189–1212. [Google Scholar] [CrossRef]
  43. Sigler, T.; Neal, Z.; Martinus, K. The brokerage roles of city-regions in global corporate networks. Reg. Stud. 2023, 57, 239–250. [Google Scholar] [CrossRef]
  44. Ma, H.; Fang, C.; Pang, B.; Wang, S. Structure of Chinese city network as driven by technological knowledge flows. Chin. Geogr. Sci. 2015, 25, 498–510. [Google Scholar] [CrossRef]
  45. Sun, W.; Zhao, Y.; Sun, L. Big data analytics for venture capital application: Towards innovation performance improvement. Int. J. Inf. Manag. 2020, 50, 557–565. [Google Scholar] [CrossRef]
  46. Haner, U. Innovation quality—A conceptual framework. Int. J. Prod. Econ. 2002, 80, 31–37. [Google Scholar] [CrossRef]
  47. Higham, K.; de Rassenfosse, G.; Jaffe, A. Patent quality: Towards a systematic framework for analysis and measurement. Res. Policy 2021, 50, 104215. [Google Scholar] [CrossRef]
  48. Singh, J. Distributed R&D, cross-regional knowledge integration and quality of innovative output. Res. Policy 2008, 37, 77–96. [Google Scholar] [CrossRef]
  49. Chen, C.; Lin, B.; Lin, J.; Hsiao, Y. Technological diversity, knowledge flow and capacity, and industrial innovation. Technol. Anal. Strateg. Manag. 2018, 30, 1365–1377. [Google Scholar] [CrossRef]
  50. Hu, J.; Pan, X.; Huang, Q. Quantity or quality? The impacts of environmental regulation on firms’ innovation-Quasi-natural experiment based on China’s carbon emissions trading pilot. Technol. Forecast. Soc. Change 2020, 158, 120122. [Google Scholar] [CrossRef]
  51. Liu, S.; Du, J.; Zhang, W.; Tian, X.; Kou, G. Innovation quantity or quality? The role of political connections. Emerg. Mark. Rev. 2021, 48, 100819. [Google Scholar] [CrossRef]
Figure 1. The interaction within multilevel networks.
Figure 1. The interaction within multilevel networks.
Systems 12 00057 g001
Figure 2. Theoretical framework.
Figure 2. Theoretical framework.
Systems 12 00057 g002
Figure 3. Organizational collaboration network in 2018–2020 period.
Figure 3. Organizational collaboration network in 2018–2020 period.
Systems 12 00057 g003
Figure 4. Knowledge network in 2018–2020 period.
Figure 4. Knowledge network in 2018–2020 period.
Systems 12 00057 g004
Figure 5. Urban collaboration network in 2018–2020 period.
Figure 5. Urban collaboration network in 2018–2020 period.
Systems 12 00057 g005
Figure 6. The moderating effect of SHK on the relationship between SHO and innovation quantity.
Figure 6. The moderating effect of SHK on the relationship between SHO and innovation quantity.
Systems 12 00057 g006
Figure 7. The moderating effect of SHK on the relationship between SHO and innovation quality.
Figure 7. The moderating effect of SHK on the relationship between SHO and innovation quality.
Systems 12 00057 g007
Figure 8. The moderating effect of SHU on the relationship between SHO and innovation quantity.
Figure 8. The moderating effect of SHU on the relationship between SHO and innovation quantity.
Systems 12 00057 g008
Figure 9. The moderating effect of SHU on the relationship between SHO and innovation quality.
Figure 9. The moderating effect of SHU on the relationship between SHO and innovation quality.
Systems 12 00057 g009
Figure 10. The joint moderating effect of SHK and SHU.
Figure 10. The joint moderating effect of SHK and SHU.
Systems 12 00057 g010
Table 1. Descriptive statistics and correlation matrix.
Table 1. Descriptive statistics and correlation matrix.
Variable123456789101112
1. Quantity1
2. Quality0.488 ***1
3. SHO0.374 ***0.212 ***1
4. SHK0.289 ***0.124 ***0.352 ***1
5. SHU0.071 ***0.040 ***0.075 ***0.081 ***1
6. SCA0.333 ***0.174 ***0.336 ***0.590 ***−0.0171
7. AGE0.172 ***0.076 ***0.211 ***0.303 ***0.0030.497 ***1
8. KS0.466 ***0.211 ***0.433 ***0.531 ***0.088 ***0.583 ***0.373 ***1
9. DC0.397 ***0.130 ***0.311 ***0.323 ***0.068 ***0.192 ***0.074 ***0.396 ***1
10. DO0.061 ***−0.066 ***0.025 **0.089 ***0.108 ***0.0200.078 ***0.191 ***−0.043 ***1
11. DK0.059 ***−0.071 ***0.027 **0.094 ***0.112 ***0.021 *0.085 ***0.198 ***−0.024 *0.425 ***1
12. DU−0.0050.021 *−0.009−0.006−0.031 **0.002−0.011−0.019−0.057 ***0.079 ***−0.273 ***1
Mean143.8631.6851.0250.9771.5415.35511.8404.0850.0050.0060.0430.135
SD640.61617.8340.2140.4740.3092.5118.2261.9250.0120.0010.0130.011
Note: *** p < 0.01, ** p < 0.05, and * p < 0.1.
Table 2. Results of negative binomial regression.
Table 2. Results of negative binomial regression.
Innovation QuantityInnovation Quality
Model 1Model 2Model 3Model 4Model 5Model 6Model 7Model 8
SHO0.113 ***
(0.015)
0.167 ***
(0.019)
0.117 ***
(0.016)
0.171 ***
(0.019)
0.155 ***
(0.040)
0.209 ***
(0.074)
0.172 ***
(0.042)
0.206 ***
(0.073)
SHO2 0.057 ***
(0.014)
0.003
(0.031)
0.037 **
(0.016)
−0.013
(0.031)
SHK 0.031
(0.026)
0.023
(0.026)
0.341 ***
(0.079)
0.347 ***
(0.080)
SHO × SHK −0.079 ***
(0.017)
−0.082 ***
(0.017)
−0.116
(0.077)
−0.098
(0.079)
SHO2 × SHK 0.071 **
(0.029)
0.071 **
(0.031)
SHU 0.054 ***
(0.015)
0.049 ***
(0.016)
−0.032
(0.039)
−0.048
(0.048)
SHO × SHU −0.029 *
(0.018)
−0.036 *
(0.022)
−0.049
(0.047)
0.029
(0.080)
SHO2 × SHU 0.054 ***
(0.019)
0.088 **
(0.037)
SHK × SHU 0.027
(0.017)
0.029
(0.055)
SHO × SHK × SHU 0.014
(0.020)
−0.038
(0.086)
SHO2 × SHK × SHU 0.041 *
(0.038)
SCA0.353 ***
(0.020)
0.350 ***
(0.020)
0.359 ***
(0.020)
0.358 ***
(0.020)
0.457 ***
(0.054)
0.438 ***
(0.054)
0.459 ***
(0.054)
0.443 ***
(0.054)
AGE−0.144 ***
(0.015)
−0.138 ***
(0.015)
−0.141 ***
(0.015)
−0.136 ***
(0.015)
−0.091 **
(0.041)
−0.099 **
(0.041)
−0.096 **
(0.041)
−0.104 **
(0.041)
KS1.621 ***
(0.021)
1.593* **
(0.030)
1.612 ***
(0.021)
1.589 ***
(0.030)
1.362 ***
(0.056)
1.120 ***
(0.074)
1.360 ***
(0.057)
1.119 ***
(0.074)
DC−0.014
(0.015)
0.001
(0.015)
−0.013
(0.015)
0.001
(0.015)
−0.045 *
(0.025)
−0.046 *
(0.025)
−0.046 *
(0.025)
−0.046 *
(0.025)
DO−0.090
(0.099)
−0.087
(0.099)
−0.087
(0.099)
−0.082
(0.099)
1.136 **
(0.544)
1.026 *
(0.539)
1.124 **
(0.544)
1.024 *
(0.538)
DK0.118
(0.104)
0.116
(0.104)
0.113
(0.104)
0.106
(0.104)
−3.248 ***
(0.625)
−3.140 ***
(0.617)
−3.236 ***
(0.626)
−3.139 ***
(0.617)
DU0.169 ***
(0.052)
0.172 ***
(0.052)
0.164 ***
(0.052)
0.166 ***
(0.05)
0.204
(0.200)
0.274
(0.201)
0.219
(0.200)
0.289
(0.200)
YearYesYesYesYesYesYesYesYes
_cons3.046 ***
(0.037)
3.070 ***
(0.037)
3.043 ***
(0.037)
3.067 ***
(0.037)
−2.046 ***
(0.146)
−2.063 ***
(0.147)
−2.031 ***
(0.147)
−2.060 ***
(0.147)
N63316331633163316331633163316331
LR chi29260.769287.049276.579304.102713.282744.232721.822753.27
Log likelihood−25,569.34−25,556.20−25,561.43−25,547.67−4008.36−3992.89−4004.09−3988.37
Note: Robust standard errors are in parentheses. *** p < 0.01, ** p < 0.05, and * p < 0.1.
Table 3. Results of robustness tests.
Table 3. Results of robustness tests.
Innovation QuantityInnovation Quality
Model 1Model 2Model 3Model 4Model 5Model 6Model 7Model 8
SHO0.035 ***
(0.012)
0.095 ***
(0.024)
0.040 ***
(0.012)
0.103 ***
(0.023)
0.140 ***
(0.042)
0.212 ***
(0.081)
0.149 ***
(0.044)
0.211 ***
(0.081)
SHO2 0.035 **
(0.015)
0.008
(0.029)
0.015
(0.017)
0.005
(0.030)
SHK −0.019
(0.024)
−0.027
(0.024)
0.290 ***
(0.093)
0.299 ***
(0.092)
SHO × SHK −0.084 ***
(0.021)
−0.091 ***
(0.021)
−0.119 *
(0.080)
−0.104
(0.076)
SHO2 × SHK 0.044 *
(0.026)
0.038
(0.026)
SHU 0.109 ***
(0.013)
0.102 ***
(0.015)
−0.054
(0.067)
−0.063
(0.075)
SHO × SHU −0.037 ***
(0.012)
−0.053 *
(0.032)
−0.019
(0.047)
0.057
(0.064)
SHO2 × SHU 0.051 **
(0.021)
0.072 **
(0.034)
SHK × SHU 0.015
(0.017)
0.021
(0.064)
SHO × SHK × SHU 0.027
(0.028)
−0.061
(0.065)
SHO2 × SHK × SHU 0.021 *
(0.030)
SCA0.073 ***
(0.019)
0.071 ***
(0.019)
0.089 ***
(0.019)
0.088 ***
(0.019)
0.374 ***
(0.064)
0.360 ***
(0.065)
0.373 ***
(0.063)
0.361 ***
(0.065)
AGE−0.101 ***
(0.013)
−0.098 ***
(0.013)
−0.098 ***
(0.013)
−0.095 ***
(0.013)
−0.052
(0.090)
−0.063
(0.089)
−0.058
(0.090)
−0.069
(0.090)
KS2.124 ***
(0.019)
2.146 ***
(0.027)
2.101 ***
(0.019)
2.127 ***
(0.027)
1.497 ***
(0.077)
1.298 ***
(0.103)
1.500 ***
(0.080)
1.302 ***
(0.104)
BC0.004
(0.007)
0.016 **
(0.006)
0.004
(0.007)
0.015 **
(0.006)
0.016
(0.021)
0.018
(0.020)
0.017
(0.021)
0.020
(0.020)
ADO0.006
(0.014)
0.005
(0.014)
0.008
(0.014)
0.007
(0.014)
0.018
(0.054)
0.020
(0.055)
0.020
(0.054)
0.022
(0.054)
ADK−0.090 ***
(0.019)
−0.086 ***
(0.019)
−0.083 ***
(0.019)
−0.079 ***
(0.019)
1.021 ***
(0.082)
1.012 ***
(0.084)
1.009 ***
(0.076)
0.999 ***
(0.078)
ADU0.039 **
(0.018)
0.040 **
(0.018)
0.035 *
(0.018)
0.035 **
(0.018)
0.628 ***
(0.076)
0.652 ***
(0.076)
0.636 ***
(0.076)
0.662 ***
(0.076)
_cons2.109 ***
(0.013)
2.130 ***
(0.014)
2.107 ***
(0.013)
2.128 ***
(0.014)
−2.320 ***
(0.082)
−2.356 ***
(0.090)
−2.312 ***
(0.082)
−2.363 ***
(0.091)
N63316331633163316331633163316331
Wald chi230,479.4932,774.7631,128.2833,209.171931.121933.471968.111981.42
Log likelihood−20,756.14−20,741.07−20,718.75−20,702.24−4083.27−4073.09−4079.19−4068.48
Note: Robust standard errors are in parentheses. *** p < 0.01, ** p < 0.05, and * p < 0.1.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Zhao, Y.; Li, Q.; Lyu, J. Quantity or Quality? The Impact of Multilevel Network Structural Holes on Firm Innovation. Systems 2024, 12, 57. https://doi.org/10.3390/systems12020057

AMA Style

Zhao Y, Li Q, Lyu J. Quantity or Quality? The Impact of Multilevel Network Structural Holes on Firm Innovation. Systems. 2024; 12(2):57. https://doi.org/10.3390/systems12020057

Chicago/Turabian Style

Zhao, Yan, Qiuying Li, and Jianlin Lyu. 2024. "Quantity or Quality? The Impact of Multilevel Network Structural Holes on Firm Innovation" Systems 12, no. 2: 57. https://doi.org/10.3390/systems12020057

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop