Performance Implications of Organizational and Technological Innovation: An Integrative Perspective
Faculty of Technical Sciences, University of Novi Sad, 21102 Novi Sad, Serbia
*
Author to whom correspondence should be addressed.
Sustainability 2022, 14(5), 2836; https://doi.org/10.3390/su14052836
Submission received: 30 December 2021
/
Revised: 23 February 2022
/
Accepted: 23 February 2022
/
Published: 28 February 2022
(This article belongs to the Special Issue Revisiting the Impact of Technological and Organisational Innovation)
Abstract
:Manufacturing firms engage in various innovation activities to achieve a sustainable competitive advantage. Although technological innovation is considered one of the key performance drivers, organizational innovation has become increasingly prominent. This study analyzes the relationship between organizational and technological innovation and examines their effect on firm performance in the manufacturing context. The empirical evidence is based on the European Manufacturing Survey (EMS)—Serbian data set gathered in 2018. Hypotheses on the relationship between organizational and technological innovations and firm performance are tested by structural equation modeling using data from 240 Serbian manufacturing firms. The results have shown that technological concepts strongly mediate the impact of organizational factors on firm performance (return on sales—ROS). Moreover, it could be said that, in contrast to the individual approach, to foster a ROS, a synergic effect of organizational and technological concepts is needed, implying the need for an integrative perspective in the process of innovation.
1. Introduction
The manufacturing environment is characterized by a constant demand for novelty for producing superior outcomes. Bearing in mind the importance of innovation as a constitutive part of the growth paradigm [1], practitioners and scholars have long recognized the importance of innovation in driving firm performance. Innovation, according to Crossan and Apaydin (2010), includes the “production or adoption, assimilation, and exploitation of a value-added novelty in economic and social spheres; renewal and enlargement of products, services, and markets; development of new methods of production; and establishment of new management systems” [2]. Thus, innovation is a multifaceted concept that includes the generation, development, and implementation of an idea or behavior, which is novel to a respective organization and conducive to superior performance [3].
The previously under-researched topic in the innovation body of literature, i.e., the complementarity effect of innovation activities, has raised heated academic discussions [4,5,6,7]. Moreover, there are many avenues to advance our understanding in this domain, especially regarding the relationship between technological and non-technological innovation and their relevance as drivers of a firm’s financial performance [8]. This paper aims to provide a contribution to the discussion of such research questions as: What is the relationship between organizational and technological innovation and what are the effects of their implementation on a manufacturing firm’s performance? Following these debated concerns, this study contributes to the innovation–firm performance literature in two main ways. First, this study extends the literature on organizational and technological innovation and their individual/complementary effect on firm performance by demonstrating the relationship and nature of their combined implementation in the manufacturing sector. Second, this study contributes to the findings related to the innovation activities and subsequent performance outcomes in the context of emerging and developing economies, which are yet to establish the most prolific innovation strategies or set up adequate innovation portfolios. Although the empirical evidence presented in this paper suggests that individual implementation of organizational and technological innovation does not significantly impact financial performance, the results are indicative of their complementarity effect which could lead, eventually, to higher performance. Specifically, digital factory technologies and automation and robotics, strongly mediate the positive effects of the organization of production and management/controlling, on a firm’s financial performance.
The paper is structured as follows. The following section introduces the literature review, including the research hypothesis. Section 3 describes the methodology, data, and research model. The data analysis and research results are presented in Section 4 as well as the theoretical and practical implications, research constraints, and future research directions. Finally, the conclusion provides an overview of the research paper.
2. Literature Review and Hypothesis Development
While questing for best practices in applying innovation drivers and producing desirable outcomes, including innovation factors interdependencies, researchers have followed numerous, both convergent and divergent, research directions. The complex nature of the innovation phenomenon has raised academic opinions, reflected in terminological discrepancies of certain innovation types. Thus, such a state of affairs has led prominent researchers to use different terms to denote the same/similar innovation concepts and to try to conceptualize the different innovation types [9] adequately. Eventually, the growing importance of practical implications of this subject has resulted in an inductive and converging emergence of the OECD innovation framework, and to a definition of different innovation types, differentiating between (1) product innovation, (2) process innovation, (3) marketing innovation, and (4) organizational innovation. Hence, according to the Oslo Manual (2007), “an innovation is the implementation of a new or significantly improved product (good or service), or process, a new marketing method, or a new organizational method in business practices, workplace organization or external relations” [10]. The former two may be classified as technological, and the latter ones as non-technological innovations. The prominent studies have mainly adopted the Oslo Manual recommendations and have extensively implemented this perspective in their analysis and innovation conceptualization [3,11,12].
Numerous definitions and innovation typologies [3,13] delineate technological and organizational concepts to distinguish between a firm’s technological and organizational, management systems [14]. The concept of technological innovation addresses the adoption of a new product, service, or new components related to the production process [15]. On the other hand, Birkinshaw et al. (2008) refer to organizational innovation as “the generation and implementation of a management practice, process, structure, or technique that is new to the state of the art and is intended to further organizational goals” [16]. Over time, authors have conceptualized the distinctive phenomenon of organizational innovation from different perspectives, making the present research somewhat fragmented [16]. The term “administrative innovation“ was used by Daft (1978) and Damanpour (1991) [9,17], although the terms “management/managerial innovation” [18,19,20,21,22,23] and “organizational innovation” [8,16,24,25] are prevalent in recent literature, and although these notions should not be viewed as completely interchangeable as they carry some specific implications, they do overlap extensively. Nevertheless, they are used to differentiate from technological innovations [14].
The manufacturing imperative to accelerate and intensify innovation has contributed to the viewpoint of the paradigm-changing technologies as key enablers of the enhanced value proposition. Amid the current manufacturing paradigm shifts and the advances in the technology realm, this perspective becomes further reinforced; thus, scholars are of the viewpoint that the industry has remained techno-centric [26,27,28], with a research focus on the technological process innovation [29]; however, to achieve a sustainable competitive advantage, firms should introduce non-technological innovation, especially considering the constraints in their replication [30].
Laforet (2016), Armbruster et al. (2008), and Ballot et al. (2015) have recently made substantial contributions in the attempt to integrate additional insights into the concept of organizational innovation [25,31,32]. Special attention was given to the relationship between non-technological and technological innovation [33], especially regarding the potential facilitating role of organizational innovation in the implementation of technological ones [16,34]. Previous research findings suggest that adopting new organizational concepts has favorable effects on the development of technological innovations and recommend refocusing on both innovation types to achieve more efficient coupled effects [35]. Although Gunday et al. (2017) and Cozzarin et al. (2019) challenge this notion [11,36], Hollen et al. (2013) and Mothe and Thi (2010) agree that organizational and technological innovation reinforce the implementation of each other [33,37].
The relationship dynamics of the organizational and technological components of innovation is a complex and multifaceted concept [5]. The technological component of the innovation process is considered highly relevant in manufacturing; however, additional aspects need to be integrated into comprehensive studies. According to Damanpour and Aravind (2012), there is a dire need to research beyond the analysis of the technological realm, as there are positive indications of the importance of both innovation types [38]. The induction of new technologies into the firm environment has to be followed by organizational change and redesign that involves employee adaption to a changing environment [39,40]. Adequate innovation adoption is an important part of the innovation process [41]. Coupling with knowledge resources acquired by the company is expected to have a favorable effect on innovation implementation [42]. The knowledge-sharing culture leverages innovation performance [43,44]; hence, the organizational environment is an important prerequisite for innovation adoption, but the lack of its individual factors can represent a barrier to successful innovation adoption [45]. Furthermore, the introduction dynamics of organizational innovation has an impact on technological innovation persistence [46], as previous innovation activities enhance the outcomes of future innovation activities [47].
Torres and Augusto (2020) and Arranz et al. (2019) were, among many, intrigued by the potential complementarity effects of the implementation of organizational and technological innovation [4,42]. Hollen et al. (2013) and Battisti and Stoneman (2010) have conducted extensive research to investigate the positive effects of the combined implementation of these concepts [37,48]. Prior studies argue that technological and organizational concepts are intertwined in a way that positive performance implications are reflective of both innovation types, indicating the complementarity effect of their implementation [3,8,48,49]. Lee et al.’s (2019) research findings imply synergic effects between technological and non-technological innovation [50]. Hervas-Oliver et al. (2018) argue that practitioners should have a comprehensive view of the innovation process that allows the integration of these two innovation types in terms of their performance effects [51]. Chen et al. (2020) state that organizational innovation has an antecedent and moderating role in implementing technological innovation, affecting firm performance [52]. Azar and Ciabuschi (2017) and Camison and Villar-López (2014), in their research of the innovation-performance stream, also argue that organizations need to leverage their organizational innovation capabilities to achieve higher performance outcomes [5,49].
Different criteria are applied in measuring firm performance in terms of efficiency and competitiveness. Financial measures, such as return on sales (ROS), are frequently employed [11,53]. In this research, ROS serves as a proxy for measuring firm performance and indicates return on sales before tax for 2017.
Keeping in mind the complexity of this under-investigated research topic [8] and the scarcity of empirical literature regarding the relationship among different innovation types [11], we have formulated the following hypothesis in an attempt to shed some empirical light on the aforementioned research questions (Table 1).
3. Materials and Methods
3.1. Data and Sample
The research findings are based on data from the European Manufacturing Survey (EMS), specifically the data set of the Republic of Serbia, obtained in 2018 [54]. The EMS is carried out in the 17 European countries and coordinated by the Fraunhofer ISI from Germany [55]. The research has targeted firms with 20+ employees in the manufacturing sector (NACE Rev 2 codes 10 to 33). The response rate was 34% (240 responders). The data gathered through the survey fully corresponds to the population, since it is the result of the stratified random sampling based on the firms’ sector, region, and size. The survey was conceptualized to include a set of core questions and a set of country-specific questions. The questions pertaining to organizational innovation are categorized in subsets that address the following organizational concepts: organization of production, management/controlling, and human resources. On the other hand, the implementation of technological innovation is surveyed through the subset of questions focusing on technologies related to the concepts of digital factory, automation and robotics, additive manufacturing, and energy efficiency.
This study used a factor structure within organizational and technological concepts “as is”, as proposed by the Fraunhofer ISI Institute. Such an approach excluded the need to run exploratory factor analysis (EFA) to compose factor structure in the first place. Moreover, to the best of the authors’ knowledge, the EMS questionnaire’s exploratory nature and item’s structure undermine the very purpose of the EFA application, keeping in mind that previous EMS studies did not provide substantial empirical evidence for validating the proposed factor structure.
The EMS questionnaire’s pre-defined factor structure presumed that the organizational concepts were represented by ‘Organization of production’, ‘Management/controlling’, and ‘Human resources’. The technological concepts consisted of ‘Digital factory’, ‘Automation and robotics’, ‘Additive manufacturing technologies’, and ‘Energy efficiency technologies’. The factor structure with cofounding items is given in Table 2.
3.2. Reliability and Validity Assessment
The reliability analysis by calculating the Cronbach’s alpha coefficients was omitted for two main reasons. Firstly, the usual approach of using Cronbach’s in similar studies where a questionnaire is a measurement instrument was not suitable, given that the scales were measured on a categorical, three-point (1-low, 2-medium, 3-high), rather than on Likert five-point scale [56]. The second reason is the lack of pre-defined factor structure empirical confirmation. Organizational and technological concepts, along with their sub-structure of factors and corresponding items, were not, as stated before, extensively empirically validated in previous studies in the first place. Therefore, rather than “confirm” an unconfirmed structure, this study is more about the plausibility of using structural equation modeling (SEM) in EMS studies and “exploring” the general context of EMS research dimensions; however, this does not exclude calls for future studies, where empirical validation of EMS factors structure should occur and be carefully examined and discussed with a broader academic audience.
For validity testing of the measurement model, the confirmatory factor analysis (CFA) was conducted by convergent and discriminant validity techniques in Mplus [57]. Given the very nature of categorical variables, the weighted least square mean and variance—WLSMV—extraction method was applied [57].
Convergent validity measures item cohesiveness concerning its underlying construct. For this purpose, validity was assessed by the Tucker–Lewis index (TLI) [58]. The convergent validity assessment values were adequate, following the notable statistical significance of factor loadings for most manifest variables (t ≥ 1.96) [57]. Following the literature recommendations, the manifest variables with non-statistical factor loadings were excluded from the model (u14 and u21). Factor loadings and their statistical significance are shown in Table 3.
Discriminant validity represents the distinctiveness between sets of indicators with respect to their corresponding factors. Accordingly, the difference between the chi-square values for pairs of constructs must be statistically significant (p ≤ 0.01) [59]. This criterion was met for all the constructs.
3.3. Structural Modeling
Evaluation of the structural model was conducted following several indices for the goodness of fit. These are the chi-square, root-mean-square error of approximation—RMSEA, comparative fit index—CFI, and Tucker–Lewis index—TLI. Yielded values were examined and compared with fit statistics recommendations [57,60]. The results were in the range of recommended values. The structural model path coefficients between the organizational and technological concepts are given in Table 4, while the effects of latent variables on the outcome (return on sales) are shown in Table 5. Structural model fit statistics are given in Table 5. An empirically tested model with path coefficients is presented in Figure 1.
The measurement model fit statistics are given in Table 6.
Table 4 illustrates that there is a statistically significant relationship between the organization of production and digital factory, along with the statistically significant relationship between management/controlling and automation and robotics (α ≤ 0.01). In addition, there is also a statistically significant relationship between the organization of production, human resources, and energy efficiency technologies (α ≤ 0.05); however, energy efficiency technologies do not produce a statistically significant positive impact on return on sales. Further, as shown in Table 5, the positive impact of the organization of production and management/controlling on ROS is fully mediated by the digital factory, automation, and robotics.
The results have shown that an organization of production has a strong, statistically significant, positive impact on digital factory, along with a strong, statistically significant, positive impact of management/controlling on automation and robotics. This impact is further fostered, followed by the strong, statistically significant, positive impact of the digital factory and automation and robotics on return on sales. Therefore, it could be said that technological concepts, i.e., digital factory and automation and robotics, have a strong and full, mediating role between organizational concepts and return on sales.
4. Results and Discussion
This paper analyzed the effects of organizational and technological innovations in manufacturing firms, on business performance measured by ROS. The organizational concepts that represent innovations in the surveyed firms and their percentage of implementation are presented in Table 7. The results have shown that, in 2018, 64.6% of Serbian manufacturing firms had implemented innovations that entailed a standardization of work instructions, followed by 63.3% of organizations that introduced on-the-job training and 62.9% that obtained certified quality standards. These results indicate that most firms had realized the significance of organizational concepts related to efficient business processes, as they had implemented at least one organizational innovation in the respective period.
However, organizational innovation, per se, does not directly positively impact the return on sales. Such findings contrast previous studies from developed economies, which speaks in favor of a positive effect between organizational innovation and firm performance [11,49]. Moreover, studies from emerging economies imply a shortcoming effect of non-technological ones (i.e., organizational concepts) on firm performance. Further, given that their impact varies across different sectors, such results may imply the need for diverse and context-sensitive innovation strategies [3].
Table 8 data illustrates the percentage of the manufacturing firms that implemented technological innovations, categorized as digital factory technologies, automation and robotics, additive manufacturing technologies, and energy efficiency technologies. Accordingly, the highest level of technological innovation pertained to the concept of the digital factory, i.e., the implementation of software for production planning and scheduling (40%), digital data exchange with customers and suppliers (33.3%), and near real-time production control systems (28.8%). Overall, the extent of technological concepts is not as prevalent as the organizational ones and although technologies are at the peak of Industry 4.0, emerging economies have not fully grasped the advanced technological concepts [61]. This study further supports such claims that technological innovations in emerging economies do not necessarily produce positive outcomes on ROS.
By testing Hypotheses 1–4, we analyzed the relationship between organizational and technological innovation on ROS. The research findings partially support Hypothesis 1 and 2, as the implementation of technological innovation, i.e., digital factory technologies, and automation and robotics, fully mediates the positive effects of organizational innovation, i.e., the organization of production and management/controlling, on firm performance (ROS) (Figure 1). Precisely, Hypothesis 1a is fully supported by empirical evidence, since the implementation of technologies related to digital factory mediates the positive effects of the organization of production on firm performance. Along with this, Hypothesis 2b is fully supported, as the implementation of automation and robotics mediates the positive effects of management/controlling on firm performance. On the other hand, the findings do not support Hypotheses 3 and 4 since there is no mediation effect of additive manufacturing technologies and energy efficiency in a relationship between the organizational concepts and ROS. This aligns with previous studies that advocate synergic complementarity effects of organizational and technological innovation on firm performance [52]. The findings of this study partially corroborate previous findings of an unfavorable individual implementation of innovation types [14], suggesting that organizational and technological innovations should be coupled in producing desirable results in a firm’s performance [37,48]. In that regard, decision-makers and practitioners should look for more viable approaches to increase financial performance by fostering organizational innovation through the enablers of technological concepts [34,49], especially in the manufacturing context of developing economies. At the same time, they can struggle with the potential lack of financial resources and this calls for future studies regarding the financial fostering of organizational and technological innovations, eventually leading to higher concept adoption rates [49].
4.1. Theoretical Implications
This study contributes to the body of literature regarding relationships between organizational and technological innovations and their effects on firm performance in manufacturing industries. The results indicate that the individual implementation of organizational and technological innovations does not positively impact financial performance. Rather, practitioners should favor a coupled approach. Therefore, to produce higher performance results and positive effects on financial performance (ROS), the implementation of organizational concepts (i.e., organization of production and management/controlling) should be fully mediated by technological ones (digital factory technologies and automation and robotics).
4.2. Practical Implications
The results of this research offer guidelines for practitioners and policymakers, especially those in emerging economies. When shaping innovation strategies, they face additional financial constraints to better allocate financial resources in both organizational and technological innovations since they have a complementary effect on performance outcomes. To achieve higher financial outcomes, decisions regarding the implementation of specific innovations should gravitate towards the investments in digital factory and automation and robotics in the technological realm, and their organizational counterparts, organization of production and management/controlling. Moreover, widely adopted digital factory concepts like software for production planning and scheduling, a digital exchange of data, and near real-time production control systems could be the favorable ones that produce desirable outcomes between organizational concepts and firms’ performance. Subsequently, in the context of an organization of production and management/controlling, predominant actions are standardized work instructions and certified quality standards. Such claims are supported by a substantial literature backbone [40]. In addition, on-the-job training is one of the most implemented organizational innovations among Serbian manufacturing firms. Since its implementation substantially impacts firms’ knowledge resources, such concepts might further advance the overall innovational process. [41,42].
4.3. Limitations and Future Research
As highlighted in the literature review, innovation typology encompasses several innovation types; however, this research is limited to particular ones only related to organizational and technological concepts. Therefore, future studies should include a more comprehensive overview of the innovational effects on firm performance outcomes, including the conceptualization and empirical validation, to shed more light on imposed research questions. Furthermore, such studies should be conducted with respect towards revealing the importance of contextual factors (sector-level analysis, technology intensity, firm size, and age) and organizational culture characteristics [8,45]. Temporal aspects of such studies could play a significant role in the relationship dynamics between organizational and technological innovations [47]. Therefore, the adoption maturity of organizational and technological innovations among Serbian manufacturing firms may pose as a groundwork for additional research and advancements.
5. Conclusions
This study reveals the nature of the relationship between organizational innovations and firms’ financial performance regarding the mediating role of technological innovations. The study is based on a European Manufacturing Survey conducted in the Republic of Serbia in 2018. The findings suggest that individual implementation of organizational innovation and technological concepts does not significantly impact firms’ financial performance. Rather, a coupled approach in emerging economies is a favorable one. Moreover, in contrast to the belief that organizational and technological concepts might stand separately in developed economies, in the context of the research population, organizational concepts such as the organization of production and management/controlling should foster technological ones, such as digital factory technologies and automation and robotics. This is reflected in the empirical evidence of higher financial performance, revealing the need for a combined effect of organizational and technological innovations. The results of this study extend previous findings in support of the synergistic and complementarity effects of different innovation types.
The empirical evidence of this study is limited to the context of emerging and developing countries, particularly considering the constrained resources for the acquisition of advanced technologies. The inclusion of other EMS populations from emerging economies could offer additional insights into the innovation dynamics and performance effects. Although the return on sales is a relevant financial indicator of firm performance, other performance measures should be taken into account, i.e., lead-time, productivity, and production quality; hence, supplementary research could yield integrative approach insights. Previous research confirmed the importance of the manufacturing industry type and firm size in innovation outcomes. Thus, future studies could examine the innovation configuration relative to these specific conditions. Additionally, it should be mentioned that the servitization trend in manufacturing is another potential avenue for future studies, given that the rising tide of studies delineates results in the manufacturing and service industries. Furthermore, the other innovation types could be considered and the other potential antecedent and moderating/mediating factors.
Author Contributions
Conceptualization, T.T., N.M. and M.D.; methodology, M.D. and T.T.; formal analysis, N.M., M.D. and T.T.; investigation, T.T., N.M., N.Z. and D.G.; writing—original draft preparation, T.T., M.D. and N.M.; writing—review and editing, T.T., N.Z. and D.G. All authors have read and agreed to the published version of the manuscript.
Funding
The results presented in this paper are part of the research within the project “Improvement of teaching processes at DIEM through the implementation of the results of scientific research in the field of Industrial engineering and management”, Department of Industrial Engineering and Management, Faculty of Technical Sciences in Novi Sad, University of Novi Sad, Republic of Serbia.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Hausstein, A.; Grunwald, A. The Proliferation of the Innovation Discourse, on the Formation, Semantics and Social Function of the Innovation Concept; Karlsruher Institut für Technologie: Karlsruhe, Germany, 2015; p. 18. [Google Scholar]
- Crossan, M.M.; Apaydin, M. A Multi-Dimensional Framework of Organizational Innovation: A Systematic Review of the Literature: A Framework of Organizational Innovation. J. Manag. Stud. 2010, 47, 1154–1191. [Google Scholar] [CrossRef]
- Geldes, C.; Felzensztein, C.; Palacios-Fenech, J. Technological and non-technological innovations, performance and propensity to innovate across industries: The case of an emerging economy. Ind. Mark. Manag. 2017, 61, 55–66. [Google Scholar] [CrossRef]
- Torres, P.; Augusto, M. Understanding complementarities among different forms of innovation. Eur. J. Innov. Manag. 2019, 23, 813–834. [Google Scholar] [CrossRef]
- Azar, G.; Ciabuschi, F. Organizational innovation, technological innovation, and export performance: The effects of innovation radicalness and extensiveness. Int. Bus. Rev. 2017, 26, 324–336. [Google Scholar] [CrossRef] [Green Version]
- Donbesuur, F.; Ampong, G.O.A.; Owusu-Yirenkyi, D.; Chu, I. Technological innovation, organizational innovation and international performance of SMEs: The moderating role of domestic institutional environment. Technol. Forecast. Soc. Chang. 2020, 161, 120252. [Google Scholar] [CrossRef]
- Hullova, D.; Trott, P.; Simms, C. Uncovering the reciprocal complementarity between product and process innovation. Res. Policy 2016, 45, 929–940. [Google Scholar] [CrossRef] [Green Version]
- Evangelista, R.; Vezzani, A. The economic impact of technological and organizational innovations. A firm-level analysis. Res. Policy 2010, 39, 1253–1263. [Google Scholar] [CrossRef]
- Damanpour, F. Organizational Innovation, A Meta-Analysis of Effects of Determinants and Moderators. Acad. Manag. J. 1991, 34, 555–590. [Google Scholar]
- OCDE. Oslo Manual, 3rd ed.; Eurostat: Luxembourg, 2007. [Google Scholar]
- Gunday, G.; Ulusoy, G.; Kilic, K.; Alpkan, L. Effects of innovation types on firm performance. Int. J. Prod. Econ. 2011, 133, 662–676. [Google Scholar] [CrossRef] [Green Version]
- Pino, C.; Felzensztein, C.; Zwerg-Villegas, A.M.; Arias-Bolzmann, L. Non-technological innovations: Market performance of exporting firms in South America. J. Bus. Res. 2016, 69, 4385–4393. [Google Scholar] [CrossRef]
- Geldes, C.; Felzensztein, C. Marketing innovations in the agribusiness sector. Acad. Rev. Latinoam. Adm. 2013, 26, 108–138. [Google Scholar] [CrossRef]
- Walker, R.M.; Damanpour, F.; Devece, C.A. Management Innovation and Organizational Performance: The Mediating Effect of Performance Management. J. Public Adm. Res. Theory 2010, 21, 367–386. [Google Scholar] [CrossRef] [Green Version]
- Damanpour, F.; Evan, W.M. Organizational Innovation and Performance: The Problem of “Organizational Lag”. Adm. Sci. Q. 1984, 29, 392. [Google Scholar] [CrossRef]
- Birkinshaw, J.M.; Hamel, G.; Mol, M.J. Management Innovation. Acad. Manag. Rev. 2008, 33, 825–845. [Google Scholar] [CrossRef]
- Azeem, M.; Ahmed, M.; Haider, S. Does organizational innovation moderate technical innovation directly or indirectly? Technovation 2021, 26, 187–206. [Google Scholar]
- Daft, R.L. Essentials of Organization Theory and Design; South Western Educational Publishing: Mason, OH, USA, 1978; p. 213. [Google Scholar]
- Hwang, A.-S. Integrating Technology, Marketing and Management Innovation. Res. Manag. 2004, 47, 27–31. [Google Scholar] [CrossRef]
- Ozturk, E.; Ozen, O. How Management Innovation Affects Product and Process Innovation in Turkey: The Moderating Role of Industry and Firm Size. Eur. Manag. Rev. 2020, 18, 293–310. [Google Scholar] [CrossRef]
- Khosravi, P.; Newton, C.; Rezvani, A. Management innovation: A systematic review and meta-analysis of past decades of research. Eur. Manag. J. 2019, 37, 694–707. [Google Scholar] [CrossRef]
- Kraśnicka, T.; Głód, W.; Wronka-Pośpiech, M. Management innovation, pro-innovation organisational culture and enterprise performance: Testing the mediation effect. Rev. Manag. Sci. 2018, 12, 737–769. [Google Scholar] [CrossRef] [Green Version]
- Zhang, Y.; Khan, U.; Lee, S.; Salik, M. The Influence of Management Innovation and Technological Innovation on Organization Performance. A Mediating Role of Sustainability. Sustainability 2019, 11, 495. [Google Scholar] [CrossRef] [Green Version]
- Simao, L.B.; Carvalho, L.C.; Madeira, M.J. Intellectual structure of management innovation: Bibliometric analysis. Manag. Rev. Q. 2021, 71, 651–677. [Google Scholar] [CrossRef]
- Koster, F. Organisational Antecedents of Innovation Performance: An Analysis Across 32 European Countries. Int. J. Innov. Manag. 2020, 25, 1–21. [Google Scholar] [CrossRef]
- Laforet, S. Organizational innovation outcomes in SMEs: Effects of age, size, and sector. J. World Bus. 2013, 48, 490–502. [Google Scholar] [CrossRef]
- Černe, M.; Kaše, R.; Škerlavaj, M. Non-technological innovation research: Evaluating the intellectual structure and prospects of an emerging field. Scand. J. Manag. 2016, 32, 69–85. [Google Scholar] [CrossRef]
- Volberda, H.W.; Van Den Bosch, F.A.; Heij, C.V. Management Innovation: Management as Fertile Ground for Innovation. Eur. Manag. Rev. 2013, 10, 1–15. [Google Scholar] [CrossRef]
- Kraśnicka, T.; Głód, W.; Wronka-Pośpiech, M. Management Innovation and Its Measurement. J. Entrep. Manag. Innov. 2016, 12, 95–121. [Google Scholar] [CrossRef] [Green Version]
- Teece, D.J. Explicating dynamic capabilities: The nature and microfoundations of (sustainable) enterprise performance. Strat. Manag. J. 2007, 28, 1319–1350. [Google Scholar] [CrossRef] [Green Version]
- Armbruster, H.; Bikfalvi, A.; Kinkel, S.; Lay, G. Organizational innovation: The challenge of measuring non-technical innovation in large-scale surveys. Technovation 2008, 28, 644–657. [Google Scholar] [CrossRef]
- Ballot, G.; Fakhfakh, F.; Galia, F.; Salter, A. The fateful triangle, Complementarities in performance between product, process and organizational innovation in France and the UK. Res. Policy 2015, 44, 217–232. [Google Scholar] [CrossRef]
- Mothe, C.; Thi, T.U.N. The link between non-technological innovations and technological innovation. Eur. J. Innov. Manag. 2010, 13, 313–332. [Google Scholar] [CrossRef] [Green Version]
- Lam, A. Organizational Innovation, The Oxford Handbook of Innovation; Oxford University Press: Oxford, UK, 2009; pp. 115–147. [Google Scholar]
- Anzola-Román, P.; Bayona-Sáez, C.; García-Marco, T. Organizational innovation, internal R&D and externally sourced innovation practices. Effects on technological innovation outcomes. J. Bus. Res. 2018, 91, 233–247. [Google Scholar]
- Cozzarin, B.P.; Kim, W.; Koo, B. Does organizational innovation moderate technical innovation directly or indirectly? Econ. Innov. New. Technol. 2017, 26, 385–403. [Google Scholar] [CrossRef]
- Hollen, R.M.; Van Den Bosch, F.A.; Volberda, H.W. The Role of Management Innovation in Enabling Technological Process Innovation: An Inter-Organizational Perspective. Eur. Manag. Rev. 2013, 10, 35–50. [Google Scholar] [CrossRef]
- Damanpour, F.; Aravind, D. Managerial Innovation: Conceptions, Processes and Antecedents. Manag. Organ. Rev. 2012, 8, 423–454. [Google Scholar] [CrossRef]
- Rehman, J.; Hawryszkiewycz, I.; Sohaib, O. Deriving high performance knowledge sharing culture (HPKSC), A firm performance & innovation capabilities perspective. In Proceedings of the 22nd Pacific Asia Conference on Information Systems (PACIS 2018)—Opportunities, Challenges for the Digital Society Are We Ready? Yokohama, Japan, 26–30 June 2018. [Google Scholar]
- Goos, M.; Arntz, M. The Impact of Technological Innovation on the Future of Work; Joint Research Centre (JRC): Ispra, Italy, 2019. [Google Scholar]
- Frambach, R.T.; Schillewaert, N. Organizational innovation adoption: A multi-level framework of determinants and opportunities for future research. J. Bus. Res. 2002, 55, 163–176. [Google Scholar] [CrossRef]
- Arranz, N.; Arroyabe, M.; Li, J.; de Arroyabe, J.F. An integrated model of organisational innovation and firm performance: Generation, persistence and complementarity. J. Bus. Res. 2019, 105, 270–282. [Google Scholar] [CrossRef]
- Attar, M.; Kang, K.; Sohaib, O. Knowledge Sharing Practices, Intellectual Capital and Organizational Performance. In Proceedings of the Annual Hawaii International Conference on System Sciences, Maui, HI, USA, 8–11 January 2019. [Google Scholar]
- Binsawad, M.; Sohaib, O.; Hawryszkiewycz, I. Factors Impacting Technology Business Incubator Performance. Int. J. Innov. Manag. 2019, 23, 1–30. [Google Scholar] [CrossRef] [Green Version]
- Ober, J. Innovation Adoption: Empirical Analysis on the Example of Selected Factors of Organizational Culture in the IT Industry in Poland. Sustainability 2020, 12, 8630. [Google Scholar] [CrossRef]
- Haned, N.; Mothe, C.; Nguyen-Thi, T.U. Firm persistence in technological innovation: The relevance of organizational innovation. Econ. Innov. New Technol. 2014, 23, 490–516. [Google Scholar] [CrossRef] [Green Version]
- Costa, J.; Teixeira, A.A.C.; Botelho, A. Persistence in innovation and innovative behavior in unstable environments. Int. J. Syst. Innov. 2020, 6, 1–19. [Google Scholar]
- Battisti, G.; Stoneman, P. How Innovative are UK Firms? Evidence from the Fourth UK Community Innovation Survey on Synergies between Technological and Organizational Innovations. Br. J. Manag. 2010, 21, 187–206. [Google Scholar] [CrossRef]
- Camisón, C.; Villar-López, A. Organizational innovation as an enabler of technological innovation capabilities and firm performance. J. Bus. Res. 2014, 67, 2891–2902. [Google Scholar] [CrossRef]
- Lee, R.; Lee, J.-H.; Garrett, T.C. Synergy effects of innovation on firm performance. J. Bus. Res. 2019, 99, 507–515. [Google Scholar] [CrossRef]
- Hervas-Oliver, J.L.; Sempere-Ripoll, F.; Boronat-Moll, C.; Alvarado, R.R. On the joint effect of technological and management innovations on performance: Increasing or diminishing returns? Technol. Anal. Strat. Manag. 2017, 30, 569–581. [Google Scholar] [CrossRef]
- Chen, Q.; Wang, C.-H.; Huang, S.-Z. Effects of organizational innovation and technological innovation capabilities on firm performance: Evidence from firms in China’s Pearl River Delta. Asia Pac. Bus. Rev. 2019, 26, 72–96. [Google Scholar] [CrossRef]
- Cheah, S.L.-Y.; Ho, Y.-P.; Li, S. Search strategy, innovation and financial performance of firms in process industries. Technovation 2021, 105, 102257. [Google Scholar] [CrossRef]
- Medic, N.; Anisic, Z.; Lalic, B.; Marjanovic, U.; Brezocnik, M. Hybrid fuzzy multi-attribute decision making model for evaluation of advanced digital technologies in manufacturing: Industry 4.0 perspective. Adv. Prod. Eng. Manag. 2019, 14, 483–493. [Google Scholar] [CrossRef]
- Manresa, A.; Bikfalvi, A.; Simon, A. Exploring the relationship between individual and bundle implementation of High-Performance Work Practices and performance: Evidence from Spanish manufacturing firms. Int. J. Ind. Eng. Manag. 2021, 12, 178–196. [Google Scholar] [CrossRef]
- Sijtsma, K. On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha. Psychometrika 2008, 74, 107–120. [Google Scholar] [CrossRef] [Green Version]
- Asparouhov, T.; Muthén, B. Exploratory structural equation modeling. Struct. Equ. Model. 2009, 16, 397–438. [Google Scholar] [CrossRef]
- Bentler, P.M.; Bonett, D.G. Significance tests and goodness of fit in the analysis of covariance structures. Psychol. Bull. 1980, 88, 588–606. [Google Scholar] [CrossRef]
- Ahire, S.L.; Golhar, D.Y.; Waller, M.A. Development and Validation of TQM Implementation Constructs. Decis. Sci. 1996, 27, 23–56. [Google Scholar] [CrossRef]
- Hu, L.-T.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
- Todorovic, T.; Lalic, B.; Majstorovic, V.; Marjanovic, U.; Tasic, N. General Readiness Assessment of Industry 4.0: Evidence from Serbian Manufacturing Industry. IFIP Adv. Inf. Commun. Technol. 2020, 59, 139–146. [Google Scholar] [CrossRef]
Figure 1.
Empirically validated SEM model, * significant at ≤0.01, ** significant at ≤0.05.
Table 1.
Research hypothesis.
| Research Hypothesis |
|---|
Hypothesis 1 (H1). The positive effects of organizational innovation on firm performance are mediated by the implementation of digital factory technologies. |
Hypothesis 1 (H1a). The positive effects of organization of production on firm performance are mediated by the implementation of digital factory technologies. |
Hypothesis 1 (H1b). The positive effects of management/controlling on firm performance are mediated by the implementation of digital factory technologies. |
Hypothesis 1 (H1c). The positive effects of human resources on firm performance are mediated by the implementation of digital factory technologies. |
Hypothesis 2 (H2). The positive effects of organizational innovation on firm performance are mediated by the implementation of automation and robotics. |
Hypothesis 2 (H2a). The positive effects of organization of production on firm performance are mediated by the implementation of automation and robotics. |
Hypothesis 2 (H2b). The positive effects of management/controlling on firm performance are mediated by the implementation of automation and robotics. |
Hypothesis 2 (H2c). The positive effects of human resources on firm performance are mediated by the implementation of automation and robotics. |
Hypothesis 3 (H3). The positive effects of organizational innovation on firm performance are mediated by the implementation of additive manufacturing technologies. |
Hypothesis 3 (H3a). The positive effects of organization of production on firm performance are mediated by the implementation of additive manufacturing technologies. |
Hypothesis 3 (H3b). The positive effects of management/controlling on firm performance are mediated by the implementation of additive manufacturing technologies. |
Hypothesis 3 (H3c). The positive effects of human resources on firm performance are mediated by the implementation of additive manufacturing technologies. |
Hypothesis 4 (H4). The positive effects of organizational innovation on firm performance are mediated by the implementation of energy efficiency technologies. |
Hypothesis 4 (H4a). The positive effects of organization of production on firm performance are mediated by the implementation of energy efficiency technologies. |
Hypothesis 4 (H4b). The positive effects of management/controlling on firm performance are mediated by the implementation of energy efficiency technologies. |
Hypothesis 4 (H4c). The positive effects of human resources on firm performance are mediated by the implementation of energy efficiency technologies. |
Table 2.
EMS questionnaire pre-defined factor structure.
| Concepts | First-Level Factor | Item | Encoded |
|---|---|---|---|
| Organizational concepts | Organization of production | Standardized and detailed work instructions | U1 |
| Measures to improve internal logistics | U2 | ||
| Fixed process flows to reduce setup time or optimize change-over time | U3 | ||
| Integration of tasks | U4 | ||
| Production controlling following the Pull principle | U5 | ||
| Management/controlling | Display boards in production to illustrate work processes and work status | U6 | |
| Methods of assuring quality in production | U7 | ||
| Certified quality standards | U8 | ||
| Certified energy management system | U9 | ||
| Methods of operation management for mathematical analyses of the production | U10 | ||
| Certified environmental management system | U11 | ||
| Human resources | Instruments to promote staff loyalty | U12 | |
| On-the-job training | U13 | ||
| Technologies | Digital factory | Mobile/wireless devices for programming and controlling facilities and machinery | U14 |
| Digital solutions to provide drawings, work schedules, or work instructions directly on the shop floor | U15 | ||
| Software for production planning and scheduling | U16 | ||
| Digital Exchange of product/process data with suppliers/customers | U17 | ||
| Near real-time production control system | U18 | ||
| Systems for automation and management of internal logistics | U19 | ||
| Product-lifecycle-management-systems (PLM) or product/process data management | U20 | ||
| Virtual Reality or simulation for product design | U21 | ||
| Automation and robotics | Industrial robots for manufacturing processes | U22 | |
| Industrial robots for handling processes | U23 | ||
| Additive manufacturing technologies | 3D printing technologies for prototyping | U24 | |
| 3D printing technologies for the manufacturing of products | U25 | ||
| Energy efficiency technologies | Technologies for recycling and re-use of water | U26 | |
| Technologies to recuperate kinetic and process energy | U27 | ||
| Outcome variable | Return on sales | U35 |
Table 3.
The Measurement model standardized results (CFA).
| Manifest Variables | Latent Variables | λ | Standard Error | t | p | Sig. |
|---|---|---|---|---|---|---|
| U1 | Organization of production | 0.842 | 0.060 | 14.104 | 0.000 | Yes |
| U2 | 0.941 | 0.031 | 30.749 | 0.000 | Yes | |
| U3 | 0.847 | 0.052 | 16.193 | 0.000 | Yes | |
| U4 | 0.747 | 0.063 | 11.898 | 0.000 | Yes | |
| U5 | 0.873 | 0.048 | 18.052 | 0.000 | Yes | |
| U6 | Management/controlling | 0.755 | 0.058 | 12.968 | 0.000 | Yes |
| U7 | 0.881 | 0.036 | 24.673 | 0.000 | Yes | |
| U8 | 0.911 | 0.045 | 20.250 | 0.000 | Yes | |
| U9 | 0.796 | 0.077 | 10.386 | 0.000 | Yes | |
| U10 | 0.779 | 0.086 | 9.052 | 0.000 | Yes | |
| U11 | 0.786 | 0.048 | 16.498 | 0.000 | Yes | |
| U12 | Human resources | 0.817 | 0.063 | 12.989 | 0.000 | Yes |
| U13 | 0.877 | 0.080 | 10.934 | 0.000 | Yes | |
| U14 * | Production control Digital factory | 0.195 * | 0.162 * | 1.202 * | 0.229 * | No * |
| U15 | 0.726 | 0.092 | 7.891 | 0.000 | Yes | |
| U16 | 0.844 | 0.076 | 11.132 | 0.000 | Yes | |
| U17 | 0.806 | 0.056 | 14.416 | 0.000 | Yes | |
| U18 | 0.868 | 0.052 | 16.672 | 0.000 | Yes | |
| U19 | 0.858 | 0.070 | 12.252 | 0.000 | Yes | |
| U20 | 0.769 | 0.079 | 9.722 | 0.000 | Yes | |
| U21 * | 0.160 * | 0.127 * | 1.265 * | 0.206 * | No * | |
| U22 | Automation and robotics | 0.704 | 0.139 | 5.058 | 0.000 | Yes |
| U23 | 0.802 | 0.154 | 5.194 | 0.000 | Yes | |
| U24 | Additive manufacturing technologies | 0.964 | 0.091 | 10.636 | 0.000 | Yes |
| U25 | 0.958 | 0.077 | 12.411 | 0.000 | Yes | |
| U26 | Energy efficiency technologies | 0.711 | 0.116 | 6.117 | 0.000 | Yes |
| U27 | 0.902 | 0.155 | 5.799 | 0.000 | Yes |
Note: * Removed manifest variables due to low path coefficients values, λ—factor loading.
Table 4.
The structural model standardized results (SEM)—from organizational to technological concepts.
Table 4.
The structural model standardized results (SEM)—from organizational to technological concepts.
| Organizational Concepts → | Technological Concepts | Γ | Standard Error | t | p | Sig. |
|---|---|---|---|---|---|---|
| Organization of production | Production control, Digital factory | 0.864 | 0.214 | 4.031 | 0.000 | Yes * |
| Management/controlling | 0.270 | 0.175 | 1.543 | 0.123 | No | |
| Human resources | 0.343 | 0.192 | 1.791 | 0.073 | No | |
| Organization of production | Automation and robotics | 0.070 | 0.407 | 0.173 | 0.863 | No |
| Management/controlling | 0.795 | 0.169 | 4.706 | 0.000 | Yes * | |
| Human resources | 0.694 | 0.425 | 1.632 | 0.103 | No | |
| Organization of production | Additive manufacturing technologies | 0.150 | 0.339 | 0.444 | 0.657 | No |
| Management/controlling | 0.498 | 0.327 | 1.522 | 0.128 | No | |
| Human resources | 0.059 | 0.320 | 0.185 | 0.657 | No | |
| Organization of production | Energy efficiency technologies | 0.749 | 0.348 | 2.152 | 0.031 | Yes ** |
| Management/controlling | 0.922 | 0.519 | 1.779 | 0.075 | No | |
| Human resources | 0.711 | 0.291 | 2.444 | 0.014 | Yes ** |
Note: * statistically significant at α ≤ 0.01; ** statistically significant at α ≤ 0.05; Γ—path coefficient; t—t statistics (statistically significant at t ≥ 1.96).
Table 5.
The structural model standardized results (SEM)—from org. and tech. concepts to return on sales.
Table 5.
The structural model standardized results (SEM)—from org. and tech. concepts to return on sales.
| Org. and Tech. Concepts → Return on Sales | β | Standard Error | t | p | Sig. |
|---|---|---|---|---|---|
| Organization of production | 0.059 | 1.358 | 0.044 | 0.965 | No |
| Management/controlling | 1.009 | 0.900 | 1.122 | 0.262 | No |
| Human resources | 0.575 | 1.664 | 0.345 | 0.730 | No |
| Production control/Digital factory | 0.743 | 0.331 | 2.238 | 0.036 | Yes ** |
| Automation and robotics | 0.724 | 0.350 | 2.068 | 0.040 | Yes ** |
| Additive manufacturing technologies | 0.249 | 0.225 | 1.108 | 0.268 | No |
| Energy efficiency technologies | 0.314 | 0.960 | 0.327 | 0.744 | No |
Note: ** statistically significant at α ≤ 0.05; β—path coefficient; t—t statistics (statistically significant at t ≥ 1.96).
Table 6.
Model fit statistics.
| Model | Baseline Χ2 | Χ2 | df | Χ2/df | RMSEA | CFI | TLI |
|---|---|---|---|---|---|---|---|
| CFA | 2124.825 * | 291.668 * | 254 | 1.5 | 0.033 | 0.979 | 0.976 |
| SEM | 2155.347 * | 323.391 * | 278 | 1.6 | 0.035 | 0.975 | 0.971 |
| Recommended | Comparison | less, the better | - | ≤2, 3 | <0.08 | >0.95 | >0.90 |
Note: * p value < 0.001; RMSEA = root-mean-square error of approximation; CFI = comparative fit index; TLI = Tucker–Lewis index.
Table 7.
Share of implemented organizational innovations in manufacturing firms.
| Organizational innovations | Number of Firms | Share in Total Sample (%) |
|---|---|---|
| Organization of production | ||
| Standardized and detailed work instructions (e.g., standard operation procedures, SOP, MOST) | 155 | 64.6 |
| Measures to improve internal logistics (e.g., value stream mapping/design, changed spatial arrangements of production steps) | 42 | 17.5 |
| Fixed process flows to reduce setup time or optimize change-over-time (e.g., SMED, QCO) | 62 | 25.8 |
| Integration of tasks (planning, operating, or controlling functions with the machine operator) | 77 | 32.1 |
| Production controlling following the Pull principle (e.g., KANBAN, Internal zero-buffer principle) | 42 | 17.5 |
| Management/controlling | ||
| Display boards in production to illustrate work processes and work status (e.g., visual management) | 77 | 32.1 |
| Methods of assuring quality in production (e.g., CIP, TQM, Six Sigma, preventive maintenance) | 103 | 42.9 |
| Certified quality standards (e.g., ISO 900xx) | 151 | 62.9 |
| Certified energy management system (e.g., EN ISO 50001) | 11 | 4.6 |
| Methods of operation management for mathematical analyses of production (e.g., regression analysis, queening models) | 23 | 9.6 |
| Certified environmental management system (e.g., EN ISO 14001) | 88 | 36.7 |
| Human resources | ||
| Instruments to promote staff loyalty (e.g., attractively designed responsibilities, offering learning opportunities, flexible working hours, child care) | 49 | 20.4 |
| On-the-job training | 152 | 63.3 |
Table 8.
Share of implemented technological innovations in manufacturing firms.
| Technological Innovations | Number of Firms | Share in Total Sample (%) |
| Digital factory | ||
| Digital solutions to provide drawings, work schedules, or work instructions directly on the shop floor | 59 | 24.6 |
| Software for production planning and scheduling (e.g., ERP system) | 96 | 40 |
| Digital exchange of product/process data with suppliers/customers (electronic data interchange, EDI) | 87 | 36.3 |
| Near real-time production control system (e.g., systems of centralized operating and machine data acquisition, MES) | 69 | 28.8 |
| Systems for automation and management of internal logistics (e.g., warehouse management systems, RFID) | 44 | 18.3 |
| Product-lifecycle-management-systems (PLM) or product/process data management | 31 | 12.9 |
| Automation and robotics | ||
| Industrial robots for manufacturing processes (e.g., welding, painting, cutting) | 34 | 14.2 |
| Industrial robots for handling processes (e.g., depositing, assembling, sorting, packing processes, AGV) | 22 | 9.2 |
| Additive manufacturing technologies | ||
| 3D printing technologies for prototyping (prototypes, demonstration models, 0 series) | 21 | 8.8 |
| 3D printing technologies for manufacturing products, components, forms, tools, etc.) | 14 | 5.8 |
| Energy efficiency technologies | ||
| Technologies for recycling and re-use of water (e.g., water recirculating system) | 31 | 12.9 |
| Technologies to recuperate kinetic and process energy (e.g., waste heat recovery, energy storage) | 21 | 8.8 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Todorovic, T.; Medic, N.; Delic, M.; Zivlak, N.; Gracanin, D. Performance Implications of Organizational and Technological Innovation: An Integrative Perspective. Sustainability 2022, 14, 2836. https://doi.org/10.3390/su14052836
AMA Style
Todorovic T, Medic N, Delic M, Zivlak N, Gracanin D. Performance Implications of Organizational and Technological Innovation: An Integrative Perspective. Sustainability. 2022; 14(5):2836. https://doi.org/10.3390/su14052836
Chicago/Turabian StyleTodorovic, Tanja, Nenad Medic, Milan Delic, Nikola Zivlak, and Danijela Gracanin. 2022. "Performance Implications of Organizational and Technological Innovation: An Integrative Perspective" Sustainability 14, no. 5: 2836. https://doi.org/10.3390/su14052836
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.
