Next Article in Journal
ExpGen: A 2-Step Vulnerability Exploitability Evaluation Solution for Binary Programs under ASLR Environment
Next Article in Special Issue
Special Issue on Industrial Management and Engineering in the Fourth Industrial Revolution
Previous Article in Journal
Influence of Soil Colloids on Ni Adsorption and Transport in the Saturated Porous Media: Effects of pH, Ionic Strength, and Humic Acid
Previous Article in Special Issue
Design and Implementation of Adaptable Self-Protection Plans for Public Buildings: A Nursing Home Case in Spain
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Dynamic Innovation Information System (DIIS) for a New Management Age

by
Sergio Gallego-García
1,*,
Diqian Ren
2,
Diego Gallego-García
1,
Salvador Pérez-García
1 and
Manuel García-García
3
1
Industrial Engineering Technologies of the International School of Doctorate, National Distance Education University (UNED), 28040 Madrid, Spain
2
Department of Mechanical and Aerospace Engineering, School of Engineering, University of Dayton, Dayton, OH 45469, USA
3
Department of Construction and Fabrication Engineering, National Distance Education University (UNED), 28040 Madrid, Spain
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(13), 6592; https://doi.org/10.3390/app12136592
Submission received: 31 March 2022 / Revised: 14 June 2022 / Accepted: 26 June 2022 / Published: 29 June 2022

Abstract

:

Featured Application

Generation of a Dynamic Innovation Information System (DIIS) for optimized planning and decision-making thanks to the dynamic evaluation of innovations over their life cycle, applying a methodology for Digital Ecosystems in the Fourth Industrial Revolution and an innovation management model based on the Viable System Model.

Abstract

Innovations are essential for global development and market dynamics. Innovation management is central to organizations for gaining adaptability and dynamic capabilities to ensure their sustainability over time. Right decisions are essential for the implementation of innovations. However, on many occasions, especially in the product development process, decisions are taken based on static analysis, qualitative criteria, questionnaires, and/or quantitative evaluations that are outdated. Moreover, many innovation developments do not consider the existing databases in their information systems of similar innovation projects, especially in the early phases of new innovations when evaluations are mainly driven by area, group, or person. Furthermore, inventions are introduced in different regions, plants, and socio-economic situations, providing different results. In this context, considering that innovations shape our current and future world, including all products and services, as well as how humans, organizations, and machines interact, the significance of the paper is clear. Therefore, it is necessary to develop an innovation management model based on the Viable System Model to cope with any potential future environment based on internal organizational capabilities. For this purpose, the paper designs a Digital Ecosystem for the Fourth Industrial Revolution (DE4.0) based on the Plan-Do-Check-Act methodology applicable to any information system consisting of a digital twin, a simulation model, databases from existing information systems, and quality management techniques. This DE4.0 provides a huge advantage for the applicability and scalability of innovations as it allows one to plan, monitor, assess, and improve. Moreover, based on the conceptual model, a generic project evaluation scheme is developed, providing a platform for innovation project management and control during the whole innovation life cycle. As a result, the research provides a scientific and practical contribution for an integrated management of innovations based on the best information and set of techniques available. Based on this framework, a supply-chain case study is developed. The results show how, depending on the intended goals, the past experiences, the evolution of the innovation, and the innovation scope, indicators can be influenced towards reaching the initial goals and reducing the innovation risks. Finally, a discussion about the potential use and role of the DE4.0 for innovation projects and the related learning process is performed.

1. Introduction

The need for innovation is imperative; however, at the same time, innovation is not easy. Innovation efforts over time have resulted in a multitude of failed innovation projects. Even huge companies that once were the forerunners and creators of whole markets had failed to stay competitive when changes occurred [1]. Nowadays, innovation is no longer conceived as a specific result of individual actions but more as a holistic view of innovation, leading to the challenge of transforming information into knowledge [2]. In this context, several countries chose to create innovation regulations that would help companies in the way they manage their systems, processes, activities, or innovation initiatives. The creation of these standards led necessitated a unified international standard, the ISO 56000 series. This seeks to reference some of the best practices among the different stakeholders in the innovation process [3]. However, this series does not provide an integrated system for managing and monitoring innovation projects [4]. Innovation is the basis for the development of human wealth. Thus, innovation management plays a fundamental role in knowledge generation, resource allocation, and investment planning. However, innovation management needs to consider organizational capabilities and goals in relation to its environment to remain competitive when facing changes [1,2,4]. Thus, continuous innovation development is needed to generate companies and maintain and improve the competitiveness of existing organizations. Therefore, an innovation strategy should be closely linked to the company’s vision and overall business strategy and based on comprehensive and relevant information, both from inside the company and from the market and the environment [5].
Research on the innovation process in and among organizations has evolved as a multidisciplinary endeavor. Most studies have focused on variables by using narrow research traditions. Therefore, there is a research gap in the study of organizational innovation [6]. Moreover, there is a lack of standards for the effective project management of innovation projects according to their type [7]. The selection of innovation projects is a dynamic decision-making process involving evaluating, deciding about, and allocating resources. In this context, there is a research gap in the quality of the decision-making process [8]. Considering that innovation does not always mean using the latest technology, innovation management techniques and tools can help companies to adapt to circumstances and meet market challenges in a systematic way [2]. According to Liberatone and Stylianou, only 14% of innovations have significant success. Several authors have shown that better management of innovation can increase the chances of success. Consequently, innovation process effectiveness must be assessed by financial and non-financial criteria at all stages [9] with both qualitative and quantitative factors. In this regard, such an approach does not exist, and therefore, there is a research and practical gap in innovation performance measurement [10]. The challenge is to adopt a new mode of thinking to master a new innovation management model [11]. Future innovation management needs to use new methods of innovation evaluation at different steps of the innovation process [11]. In this context, there is a need for the implementation of systematic measurement for innovation activities over time based on a combination of quality management and innovation management in an integrated management system [3]. As a result, the goal is to design an innovation management model capable of directing the strategy of an organization towards a predefined set of goals based on its digital intra-organizational logistics Viable System Model. The model defines the tasks for the different planning horizons as strategic, tactical, and operative. The goal of the research is to generate an approach for how to evaluate innovations based on different criteria in all potential future environment scenarios based on the conceptual model. There is also a practical gap for companies on how to design, develop, and integrate formal monitoring and tracking systems, where the new innovations and products are tracked relative to original objectives [5]. Therefore, a main goal of the paper is to develop a methodology for Digital Ecosystems in the Fourth Industrial Revolution (DE4.0). The methodology is based on databases and indicators from information systems such as ERP (Enterprise Resource Planning) systems at different levels, a digital twin of the given system such as a supply chain in which innovation will be implemented, then a simulation tool based on the digital twin to derive conclusions based on quality management techniques with the goal of better planning, monitoring, assessments, and adjustments, following a Plan-Do-Check-Act methodology.
The paper is structured with an introduction of the research topic, an overview of the related research background, a description of the methodology and materials used, followed by the conceptual model development and a modularization case study. It was developed by applying the conceptual model for a product modularization innovation. In this context, simulation serves as a tool to test the approach in different companies based on the developed Dynamic Innovation Information System (DIIS), which enables one to evaluate new innovations over their life cycle in a digital twin platform with simulation capabilities and quality techniques based on the PDCA methodology. Later, we provide a discussion about project management, the innovation life cycle, the use and role of information systems, digital twins, and simulations for the evaluation of the risks of innovations such as new products and technologies. Finally, we present an overview of the main conclusions and implications, as well as limitations, and future research areas.

2. Research Background and Framework

2.1. Innovation Framework

Humankind has always been able to realize ideas with innovations such as controlling fire, democracy, railways, the light bulb, and the development of new medicine [1]. In this regard, various scholars from social and economic fields were pioneers in advancing the concept of a knowledge-based economy and predicting the decline of an industrial manufacturing culture [2]. Knowledge plays a key role in the global economy as it is the basis of innovation [12]. In the mid-1990s, the knowledge-driven economy concept evolved as knowledge became more important than ever before, especially as applications of information and communication technologies were the drivers of the new economy [2]. In this context, investment by private companies in research and development (R&D) and by universities in research and education are crucial sources of knowledge and innovation [13]. The first definition of innovation was proposed by Schumpeter (1934). He associated it with economic development and considered it a new combination of productive resources. The conception of innovation has evolved significantly over the last 60 years. During the 1950s, innovation was considered a discrete development resulting from studies carried out by isolated researchers [2]. Moreover, innovation models have developed from a process within the firm to include factors of the external environment [4]. Nowadays, innovation is no longer considered a specific result of individual actions but more a holistic view of innovation, leading to the challenge of transforming information into knowledge [2]. As a result, technological development and innovation are at the core of the political and economic discussions as it has a qualitative nature (new products, technologies, etc.) that is also converted into quantitative measures (profits, market share, etc.) [4].
Every innovation model is based on the reduction of complexity of reality and, therefore, inevitably tends to overlook the specific factors of each innovation case [4]. In this context, the purpose of innovation management is to control and reduce the risk of the development strategy, considering that innovation is a high-risk activity [14] and that our mental models are limited [15]. The development of innovation project portfolios has attracted the attention of practitioners and researchers in the last two decades [16]. Moreover, an innovation project performance assessment can be performed in several ways, for instance, by considering technical, economic, and other factors [10]. During the 1980s and 1990s, debates over economic growth and competitiveness reoriented upon the centrality of innovation and pointed out the lack of measurement tools to transform the existing innovation theory into effective managerial management methods to link macro-approaches to the micro-level [17], i.e., from the strategic to the operative management levels. Moreover, innovation can have different outcomes depending on its application scope. One of the highest priorities should be given to gaining clarity on how to assess the outcomes of innovation. Simplification, conceptual clarification, theoretical models, and cumulative empirical work are needed [18] with an innovation measurement method depending on the innovation phase [9].
Innovation and continuous improvement are based on the company’s ability to be creative and learn [5]. However, innovation inevitably gives rise to new problems. It can cause chain reactions throughout an entire company [13]. The objective of innovative project management is to better integrate and control the risk associated with the stress that accompanies the process of innovation; it is the management of uncertainty [13]. A management and control model can only work if you manage to grasp all its processes and understand their interdependence [19]. In this context, innovation management techniques include a range of tools, techniques, and methodologies that help companies in a systematic way [2]. All the major actors agree that only a few innovation management techniques and tools (IMTs) are widely recognized, and most are unidentifiable and inaccessible to firms. Over 37% of the actors declared that most firms are not aware of the existence of IMTs, while 34% stated that few IMTs are sufficiently defined to be successfully applied within firms. All actors are convinced that new challenges coming from the knowledge-driven economy require new IMTs [2]. The current main IMTs used are project management (82%), followed by business plan development (67%), corporate intranets (66%), and benchmarking (60%) [1]. However, corporate reporting is still founded on a financial and management accounting model developed for the industrial economy and is not able to deal with today’s knowledge economy, where most corporate value creation is based on knowledge assets rather than on physical resources and financial capital [2]. Furthermore, there is a mutually reinforcing interaction between innovation management tools and knowledge creation, thus leading to a key role within strategic management [20]. In this context, Industry 4.0 creates opportunities based on using digital technologies and platforms to manage innovation [21]. One of them, the digital twin concept first presented in the NASA 2010 technology roadmap [22], has already been used as a platform to bring education and research activities together [23]. A DT model enables one to monitor, optimize, and forecast processes that support the continuous improvement process [24]. Simulations based on a digital twin model can increase organizational adaptability as what-if scenarios could be verified at the design stage through a series of simulation experiments [25]. Moreover, the literature refers to the existence of a significant set of practices in the quality field that can support and facilitate the formalization of integrated management systems, including quality and innovation management systems. Nevertheless, it appears that to ensure a correct and adequate transition to an IMS (Innovation Management System), companies must follow the guidelines presented throughout ISO 56002:2019 that reveal objective examples of implementation of certain practices, as it categorically explains the key points that should be included in the innovation processes and/or initiatives to ensure the total success of its innovation activities. Therefore, companies face a big challenge when trying to integrate both quality management and innovation management systems in a coherent, meaningful, and practical way [3]. There are also difficulties inherent in the lack of conceptualization for their practical application and their role in developing firm dynamic capabilities [26]. Techniques associated with entrepreneurship management (business models and plans, as well as simulation tools, technology transfer, and spin-offs) were the most popular, followed by cooperative and networking tools and industrial property management techniques. On the other hand, continuous improvement tools, production organization, and lean techniques were the least utilized, perhaps due to their manufacturing characteristics. The research has clear management implications. First, it calls for focused innovation management, applying sophisticated innovation management techniques and systems. Second, it outlines the contingent aspect of this application when looking for efficiency. Third, tools must be implemented considering the firm, its experience, and the sector of activity [26].

2.2. Organizational and Innovation Strategy

The corporate strategy concept was developed to provide managers with an operational method to ensure the optimal alignment with organizational goals in the long term [27]. In this context, the organizational structure of many companies is strongly based on functional areas aiming for the minimization of costs of individual processes but missing the opportunity for an optimal global process [21]. This needs to be complemented with the conflicts within a given process. For example, the logistical corporate conflict of goals comprises three conflicts of interest [28], one being the conflict between costs and service level.
Innovation strategies can only be defined in accordance with the corporate strategy and play a central role for companies in competition. The main goal of the corporate strategy is to increase the company’s value. This is achieved by identifying and implementing new potential for success. The research and development strategy, on the other hand, is implemented operationally by departments. It thus follows the innovation strategy and deals with the selection and evaluation as well as the optimized design of research and development projects. Thus, the company’s success depends not only on the choice and implementation of a suitable innovation strategy but also, above all, on an optimal interaction between the individual sub-strategies. The strategic management of innovations must be greatly upgraded and become the subject of an overall corporate strategy process to make this possible [9].

2.3. Innovation Management

Trott (2005) provides a list of seven types of innovation: organizational, management, product, process, production, commercial or marketing, and service innovations [15]. Securing the competitiveness of the company by means of successful innovations is the fundamental goal of innovation management [21]. The goal of innovation management can, therefore, also be seen as “the sustainable maximization of the benefits for the company created by innovation activity” [9]. In the practical field, those organizations with successful innovation models outperform their competitors in terms of development, financial performance, and employment, leading to more social benefits. However, innovation management is a complex task that requires skills and knowledge very different from the standard management skill set and experience [29].
The innovation process is defined as the development and selection of ideas for innovation and the transformation of these ideas into innovation. To emphasize the uncertain character of this innovation process, other authors use the innovation journey. Furthermore, in this paper, an innovation project is considered to be the innovation process of one particular innovation. Andrew and Sirkin argue that the management of an innovation project is essentially like any other business project, though it comes with more risk and uncertainty [1]. The efficiency of the innovation processes is primarily influenced by the operational processes of innovation management. The main task here is the performance, cost, and time evaluation of the various projects. Based on this, it is possible to control the innovation or project portfolio [9].
The core of innovation management is a systematic approach to the implementation of changes that should lead to the improvement of the products, processes, or position of the whole company. The innovation activity is successful only if there is an appropriate response from the market, for example, in the form of higher sales or happier customers, in the form of image strengthening and the creation of better relations with the individual groups of the company. However, at the same time, source options and financial requirements of the company owners and creditors must be respected, and the innovation activities cannot endanger the stability of the company. It is not possible to perceive innovations only as an improvement of the products that a company offers [19].
Before exploring the diversity of existing models, we look at how innovation process models developed over time. The mental model that people have of innovation has not been the same over time. The main reason for this is the changing nature of the environment in which innovation takes place. According to Rothwell (1994) [30], the innovation management models are Technology Push (1950s–mid-1960s), Market Pull (mid-1960s–early 1970s), The “Coupling” Model (early 1970s–mid-1980s), the integrated innovation process model (early 1980s–early 1990s), and the integrated, parallel, flexible, and connected model (1990s–present day). All models start with some form of idea generation or searching for ideas for innovation and selection of projects. The next step is to turn the (selected) idea into some tangible product, process, or service with steps such as development and realization. Later, the newly developed product, process, or service is going to be implemented in “the real world”. This phase is called implementation/launch. It entails the preparation of customers and marketing activities. Most authors stop here with their innovation process. However, some authors include a post-launch phase. This entails the sustaining and supporting of the innovation or even re-innovating and scaling it up. Finally, some authors include a phase for explicit learning [1].

2.4. Project Evaluation

Project evaluation is currently based on the strategic implications of innovation projects with regard to sustainability along economic, social, and ecological dimensions. Thus, the selection of proper evaluation criteria usually leads to an effective innovation project portfolio. Furthermore, project evaluation and selection imply challenges such as how to prioritize and maximize value when considering a given set of resource constraints [16]. By responding effectively to these challenges, a balanced portfolio of innovation projects aligned with organizational goals will be obtained. This is crucial to a company whose success depends on internal resources and its capabilities to adjust to the organizational environment dynamics [8].
Innovation controlling is described in particular at the controlling strategic level in relation to strategic financial ratios. In a hypercompetitive environment, the innovations of small and medium-sized companies become the crucial activity that decides their survival. Process innovation management and the evaluation of its efficiency and time are key competitive advantages in relation to big companies [19]. Innovation activity always involves a risk, which includes a constant number of factors. Their impact on the results of activities cannot be accurately calculated in advance. When choosing a particular project, it is necessary to evaluate its effectiveness, uncertainty, and risk factors [31].

3. Methodology and Materials

3.1. Methodology

In the research performed, the methodology is as follows:
  • Literature review based on search by keywords “Viable System Model”, “Organizational Strategy”, “Innovation Management”, “New Technologies and Products Decision-Making”, “Projects Evaluation for Innovations, New Products, and Technologies”, “Digital Twin for New Technologies and Products Decision-Making”, “Innovation Management Systems”, “Innovation tools”, “Information System for Innovation”.
  • Methodological approach selection: The Viable System Model (VSM) was selected for its structure for viability and System Dynamics (SD) for its ability to analyze the behavior of complex systems over time. The VSM is built on three main principles: viability, recursivity, and autonomy. Viability is a property of every system that can react to internal and external perturbations in order to maintain separate existence [32]. BEER explained that a company is also a viable system because it tries to survive [33]. Innovation management has been an increasingly covered topic in scientific and management literature over the last fifty years, as innovation is of key importance for the survival of an organization. When an organization is deeply involved with—and used to—what they are good at (core competencies), it becomes trapped in it. When the environment changes (e.g., changing consumer needs, changing regulations, etc.), organizations are not able to adapt [1]. The VSM suggests a new organizational approach that is based on the principle of recursion, according to which viable systems are themselves composed of viable systems, each having self-organizing and self-regulatory characteristics [34]. Based on this capability of information exchange, the term “Industry 4.0” was introduced by representatives of business, politics, and academics that promoted the idea of digitization together with autonomy and self-behavior of machines as an approach to strengthening the competitive power of the German manufacturing industry [35]. Self-optimization [35] and the advances of new IT, such as IoT, cloud computing, and Big Data, enable smart characteristics such as self-sensing, self-organizing, and self-adaptive [36]. This enables the smart product to self-organize its required manufacturing processes and its flow throughout the factory in a decentralized manner by sharing smart data with the CPS [37]. However, the theories on self-organized systems are not mature, and complex system research remains a hot topic [38].
  • Conceptual model design: The definition of recursion levels, management, and control tasks for a novel innovation management model for strategic, tactical, and operative levels.
  • Development of novel methodology for digital ecosystems: digital twin, simulation, quality techniques, and information systems’ databases.
  • Simulation and analysis of results: The development of various simulation models based on predefined assumptions, logic formulation, scenario extraction, and analysis of results for a given set of Key Performance Indicators (KPIs).
  • Critical reflection and outlook: Cooperation between research partners and the chosen methodological approach was set as a combination of a literature review, conceptual development, digital information ecosystems, quality management, and control techniques and simulation in a case study for different companies.

3.2. Methodological Elements and Materials

The following sources, methods and tools were used to perform the research:
  • Viable System Model (VSM): as mentioned above, the VSM is based on three main principles: viability, recursivity, and autonomy [39]. A viable system always consists of an invariant structure of five necessary and sufficient subsystems in relation to any organism or organization that is able to conserve its identity independently of its related environment [15]. VSM has been used as a methodological approach to describe and develop models to respond to industrial and social challenges, such as organizational models for companies [40].
  • Martensen and Dahlgaard argue that an extended Plan-Do-Study-Act (PDSA) loop is necessary when formulating excellent strategies and plans for innovation management [5]. As it can be implemented in any organization regardless of its size, area, or maturity, this guidance essentially conveys lessons that focus on different areas of action, seeking to facilitate the definition and establishment of objectives and performance indicators to assess the results of its innovation system. ISO 56002:2019 is based on the principles of innovation management [14]. The structure of the innovation management system should be viewed, relating it to the different clauses of the standard and the PDCA approach. This standard also refers to the need for an appropriate structure to manage the risks and uncertainties associated with the organization’s market, especially in the earliest stages of its creative processes. The structure that is adopted also allows for an alignment with some international standards, such as ISO 9001:2015 and ISO 14001:2015, enabling a more effective integration, development, implementation, maintenance, and continuous improvement of the innovation management system. The adoption of this standard, together with the innovation tools, facilitates and boosts the innovation process of any company by providing principles that allow it to structure and organize its internal innovation processes, maximize innovation efforts, and harness the knowledge and creativity of its employees and those with whom it collaborates externally and enable integration with other existing management systems [3].
  • System Dynamics (SD): This is an approach used for the analysis of complex models to identify and design successful policies [41]. For this purpose, it is increasingly applied in private organizations and public administrations [42].
  • Databases from information systems: Databases for the different periods of the case study were used. Databases included sales, procurement, production, distribution, economic, and personnel data. Databases were read and exported by the simulation software as well as extracted from the digital twin of the company in which an innovation project was completed.
  • Digital twin concept: consists of the digital twin of the organization and its related supply chain and environment as it can help to manage and monitor the innovation projects from the ideation phase up to their commercialization phase.
  • Simulation software: simulation models are mainly used to support decision-making as they help to analyze the impacts of the alternative decisions in advance [43]. For this purpose, SD software Vensim was selected to develop the case study.

4. Conceptual Model Development

4.1. Innovation Management Model Aligned with Organizational Goals

4.1.1. Innovation Management Tasks According to Planning Horizons

Given the focus of strategic foresight on the exploration and exploitation of potentialities and limits in organizing, past research has striven to establish a positive relationship between strategic and innovation [20]. Innovation is an activity that involves all levels within an organization, from the normative and strategic levels to the operational levels. Management and planning tasks are classified into strategic, tactical, and operational levels depending on the respective planning horizon. Thus, this classification was performed for the innovation management tasks, as shown in Figure 1. First, innovation has strategic tasks, i.e., tasks related to the vision of an organization towards fulfilling its purpose. Strategic management of innovation includes the definition of the innovation principles and program, the related organizational structure and investment program, and how to measure the innovation development with an innovation target system based on continuous observation and evaluation. Moreover, the physical centers where innovation takes place should be defined, and resources need to be allocated while defining which activities are to be performed inside and outside the organization, i.e., make-or-buy decisions.
Secondly, innovation management includes tactical management tasks such as defining the requirements that the organization needs to face its challenges and achieve the target results. For the purpose of acquiring these capabilities and functionalities, it is necessary to develop a procurement program and a selection of partners for the innovation life-cycle management. Furthermore, it is necessary to define the optimized layout of innovation and development centers as well as to identify the best-fit information system for innovation management. Then, all these activities need to be monitored through systematic innovation controlling based on the continuous observation and evaluation of the internal performance of innovations.
Thirdly, the innovation management model also presents operative tasks such as the scheduling of the activities to be performed as well as coordination of innovation projects up to the planning and control of activities of third companies while measuring and calculating the related Key Performance Indicators (KPIs).

4.1.2. Innovation Management Recursion Levels

When an invention is introduced in a company, an assessment concerning the impact of its introduction as an innovation within the organization must be assessed to estimate the expected effects on the whole organization, on the different areas of the organization, on the supply chain, as well as on the related environment. Therefore, in the conceptual development for innovation management, it is necessary to consider the company level, the areas that work at the same level as the innovation function, and the innovation function itself. Furthermore, within the model, the company is assumed as a viable system at the first recursion level. Thus, in this article, four recursion levels can be differentiated, as shown in Figure 2:
  • The recursion level of the company (n − 1), including other organizations or companies if it belongs to a group.
  • The recursion level of innovation (n). In the same recursion level, there is production, maintenance, finance, human resources, research and development, etc. This recursion level includes the innovation function, as any organization should consider how to define its strategy related to the development of inventions and introduction and diffusion of innovations within its organizational structure.
    The recursion level of the different innovation types such as technical process, product, organizational, management, production, commercial/marketing, and business model innovations (n + 1). This recursion level refers to the different innovation types that can be developed and implemented.
    The level of innovation type with the associated innovation activities for the different innovation types such as cutting and machining processes for the technical process innovation type (n + 2).
The tasks in the n +2 recursion level do not possess the structure of viable systems since they are the elements of innovation execution. In the research performed, innovation tasks were analyzed in detail, recursion level n. A useful example for understanding the recursion levels in a practical way is as follows: new development of a management innovation (n + 1) for a specific manufacturing process (n + 2), such as the laser cutting process (tasks of n + 2) is introduced in the production (n) system within an organization (n − 1) through the innovation (n) function. That is, an invention was developed for the management of the laser cutting process and was implemented and managed in the organization within the innovation function until it was established in the organization in a stable way in the production function.
In Figure 3, the recursion level of a company can be seen. It shows how an organization has a defined policy and mission, i.e., a purpose within society. Based on this and the continuous observation of the organization’s environment, the organization’s strategy is developed. The strategy is influenced by the internal state, while an organization has internal mechanisms to plan and control its operations. Within these operations, the innovation function is included, for which there is a specific innovation environment as well as the interconnection between the other functions of the organization as well as between the other organizational systems and elements. Therefore, a key part of this representation is that considering the innovation function as a part of a whole, i.e., the organization, it influences all other areas when introducing an innovation. Knowing that innovation is a high-risk activity in terms of its development and implementation success, it is essential to identify the risks along any innovation life cycle as early as possible to support decision-making in relation to investments in future inventions. That is, one should avoid or regulate investment in an invention, introduction, or the maintenance of the innovation within the organization if, for any reason (culture, technical, organizational, etc.), it has implications for other functions that compromise the effective and efficient realization of the organization’s purpose.

4.1.3. Innovation Management Model Applying the Viable System Model

This subsection aims to describe the systems of the innovation VSM while specifying the clusters of tasks related to Section 4.1.1. At the recursion level of innovation (n), as shown in Figure 4, it is assumed that the different innovation types will be the respective system 1, which also contains a viable system. The VSM of innovation management within an organization is described by the functions and activities performed by its five necessary systems:
  • System 5 establishes the innovation objectives, principles, and culture, and communicates them to the other management systems, systems 3 and 4.
  • System 4 observes and collects essential information from the external innovation environment, such as technological development, innovation trends, legislation, and competitors advances. Therefore, the innovation environment is mainly represented by the technological evolution of new manufacturing technologies and, in general, all agents and areas affecting the innovation ecosystem, such as case studies and their analysis of results, innovation costs for developing internally or buying from the market, new business models, and existing innovations in the market of the different types. With these and other information from the external related environment as well as with information from system 5, system 4 creates a vision of what the innovation development and activities of the organization have to become, and which are the required measures to be followed to reach that envisioned state. In addition, the vision of system 4 is validated with system 3. If the validation occurs, system 4 is responsible for making the decision, and system 3 must carry out the implementation of the changes related to the decision internally.
  • System 3 deals with maintaining the internal stability of the innovation management model while optimizing the internal resources’ utilization levels based on the information provided by system 4 about the environmental needs and requirements as well as on the information from system 1 through its communication with system 2. This system performs the operative innovation management and control activities. In addition, system 3* allows a quick response to potential alerts and undesired disruptions in the innovation process based on the monitoring function that it performs that enables system 3 to act before information flows through system 2.
  • System 2 represents the coordination function between the different innovation types in operational activities, thus enabling synergies and complementarity between innovation projects. This system collects all the information of the different innovation types and acts as a filter for system 3. In this context, while system 2 performs functions daily, the tactical system, system 3, optimizes innovation planning and control over a longer planning horizon.
  • System 1: each innovation type within the innovation ecosystem is an operational unit consisting of managerial units and the division that performs the operational tasks.
  • Environment includes all the external factors that affect the innovation management in an organization. Figure 4 shows the environment of the entire innovation area as well as of each innovation type.
As can be seen in Figure 4, the innovation management model was developed as an analogy of the VSM with the various innovation types as operational units of system 1.

4.2. Project Evaluation Scheme

After defining the generic conceptual framework, the steps, and elements for the evaluation of innovation projects are to be defined as shown in Figure 5. For that purpose, first, the organizational strategy has to be set. Later, based on identified capability gaps and the different innovation types, the innovation strategy is defined. Then, based on the strategy, the innovation project proposals are listed, and their characteristics are described. Furthermore, once all project initiatives and innovation project characteristics are available, the evaluation process consisting of both qualitative and quantitative assessment is to be initiated. For this purpose, generic, innovation type-specific, and innovation-specific key performance indicators should be defined and evaluated at the firm level, supply chain level, and related environment level. Thus, based on the assessment results and the resource constraints of the organization, the selection of innovation projects is realized. Finally, this process needs to be performed iteratively on a rolling basis.
For the qualitative assessment, scoring models for factors such as strategic alignment, maturity level, and team attitude can be applied. Moreover, quantitative KPIs for each innovation type needs to be defined. For instance, for technical process innovation, factors could be ergonomic, security, quality, availability, etc. KPIs need to be defined for each of the different levels. For instance, for the company level, KPIs for production could be defined as impacts on production volume, performance rate, production costs, production lead time, OEE, capacity utilization rate for quality or rejection rate, control, rework costs, etc.

4.3. Methodology for Digital Ecosystems for the Fourth Industrial Revolution (DE4.0)

This subsection aims to develop a blueprint framework for a Digital Ecosystem based on a multi-set of information databases, simulation, digital twins, to plan, implement, control, and monitor organizations and projects over their life cycle based on the Plan-Do-Check-Act methodology, as shown in Figure 6. The digital twin model is based on sub-models of the organization, its supply chain, and its related environment. A digital twin model and the link with information system databases are the basis for performing simulations that enable one to analyze policies based on what-if analysis and statistical techniques. Based on Figure 3, there are already digital twin models and simulations for the different functions within different types of organizations, such as procurement [47], distribution [45,48,49], production and quality [50], maintenance [51], information systems [52], sales [53], human resources [54,55,56], design, and development [57], for the planning and control of operations [52,58], for finance, improvement strategies, and strategic planning [59,60,61], as well as at the normative level [46]. Moreover, there are already models that can assess the three different levels, i.e., organization, supply chain, and related environment [51]. All these models share the capabilities of facing dynamic environments and attenuating their volatility by predicting and managing changes through enhanced adaptability. This is achieved thanks to a combination of forecasting techniques, quality management, simulation, and a connection to external databases for continuous improvement of an organization at all levels and in all areas. By doing so, any change in any part of the whole digital twin model can be identified and assessed, thus calculating the impacts and risks associated with any change and improving the adaptation capabilities of any organization applying the novel digital ecosystem to the management and improvement of existing information ecosystems.

4.4. The Dynamic Innovation Information System (DIIS)

By applying the digital ecosystem methodology to the innovation management models, the DIIS is developed, as seen in Figure 7. It shows how an innovation plan for any invention or innovation should be first developed. It can be realized by using qualitative or/and quantitative data from outside and/or within the company. From the outside, it would consist of open or private information related, for instance, to a specific technology. Within the company, it would be the databases from existing information systems as well as files within digital or physical repositories. This Plan-Phase has a strategic character, as it should ensure the alignment of the innovation plan with the goals of the organization, as well as consider all other functions and entities within the organization and the related environment to reduce innovation risks. To carry out the plan, statistical analysis based on historical data can support the development of a rationalized plan helping the later agreements and commitments with partners and other functions.
In the Do-Phase, the plan is converted into reality, changing from a strategic to an operative perspective. The innovation development needs to be monitored, data and parameters to be collected, and indicators to be introduced into the existing information systems and databases. To achieve that, a digital twin model provides the platform for innovation implementation monitoring.
In the Check-Phase, all data and information collected need to be analyzed to assess the evolution of the innovation indicators and parameters, that is, a control-oriented phase. For this purpose, quality management and control techniques and tools represent a fundamental element in identifying deviations and determining innovation risks.
Finally, in the Act-Phase, the DIIS derives measures for improving the innovation process based on the Check-Phase and the simulations of different policies as well as on statistical analysis to determine the impact of future planning after the adjustment promoted by the Act-Phase. As this step influences the policies and the risks of the innovation management and control project, it also has a strategic management aspect.
As shown in Figure 8, the initial planning of any innovation can be based on past experiences and practical evidence from databases, such as for technical, technical–economic, or brand new planning. However, for all these cases, the implications of the innovation in the target indicators must be assessed to proceed with the introduction of the invention as the innovation launch as well as its diffusion into the market. The theoretical background enables one to use the lessons learned and the innovation parameters to reinforce the innovation process based on the PDCA methodology. Moreover, during the innovation life cycle, from the concept to its withdrawal from the market or the company, innovation should be checked several times at certain milestones or in a continuous way based on the data and information of experience and lessons learned from past experiences. Later, based on this Check-Phase, the innovation plan is adjusted based on a dynamic Act-Plan that enables its adaptation to fulfil the expected goals.

5. Modularization Case Study Based on the DIIS

5.1. Goal and Methodological Steps of the Case Study

Firstly, the goal, scope, and methodology of the case study must be defined. The goal of the case study is to design a model for the quantification of the evaluation scheme parameters for a specific case study of different levels of modularization in product design and development for different companies. For each company and planning methodology, a new model is generated in which the simulation logic is adapted according to the different settings applied. For the methodological framework, the following steps will be taken:
  • Definition of the objective, scope, hypothesis, and methodology including a general description of target models and scenarios;
  • Definition of the Production System: Flow and Characteristics;
  • Definition of quantitative parameters and Key Performance Indicators (KPIs) to obtain results and compare models;
  • Determination of interrelationships among variables within the model;
  • Description of the main assumptions for the simplification of the complexity of the model;
  • Creation of digital twin models for the different planning models;
  • Linking databases and validation of the behavior of the simulation models;
  • Application of quality-control techniques;
  • Simulation and extraction of results;
  • Evaluation of the results and derivation of conclusions.

5.2. Design of the Generic Case Study

5.2.1. Structure of the Case Study: Production System Flow and Characteristics

This subsection includes the generic description and specifications of the case study. The general framework described herein applies to all specific digital twin and simulation models.
Firstly, this subsection describes the general structure of the models that are applied to all simulation models within the simulation case study. The structure is generated to provide the necessary production system flow and characteristics to answer the research question. Thus, as seen in Figure 9, the structure considers a production system within a supply chain of suppliers–production system–distributors–retailers/customers serving as a generic framework applicable for any sector. Moreover, the production system consists of technical processes, that is, the execution of the transportation, warehousing/storage, and production of finished products, as well as management processes, systems, and organizational structure from operational to strategic levels.
In summary, all models have a set of suppliers, one raw-materials warehouse, three production processes, one finished products warehouse, a set of distributors, and three retailers serving end-users, each with a certain demand. All models maintain this structure over the simulation period.

5.2.2. Key Performance Indicators

The results were calculated from the simulation for all models to evaluate the response according to the following key performance indicators:
  • Cumulated demand (# thous. products): market demand for the product;
  • Cumulated real demand (# thous. products): real demand based on the market demand minus the demand loss due to long customer order lead times;
  • Cumulated production (# thous. products): the cumulative sum of all units produced over the 500 simulated production weeks;
  • Ø Availability of the production plant (%);
  • Ø Performance at the final production step (# thous. products/week);
  • Development Time (# weeks): time required for the innovation to be implemented;
  • Absenteeism rate (%): percentage of employees that do not assist their workplace because of various motives, one being the medical conditions generated by non-ergonomic workplaces;
  • Labor productivity (products/employeex week);
  • Ø Production lead time (# weeks): the number of weeks between the placement of the order and the delivery of the product for its distribution;
  • Ø Suppliers’ quality rate (%);
  • Cumulated Service level (%): the quantity of units delivered on time divided by the total number of delivered units;
  • Sales (million USD): multiplication of the number of produced units by the sales price of each produced unit type;
  • Cumulated Operational Costs (million euros): all costs related to production system operations. It is the sum of procurement, production and distribution costs considering raw materials costs, transportation activities, working capital, labor costs, working shifts, and maintenance costs. The running costs of the project initiative are also included;
  • Cumulated Investment (million USD): the amount of the investment made to improve the production system;
  • Profits (million USD): the result of the sales minus operational costs;
  • Return on investment (ROI) (%): the difference in profits minus the investment value divided by the investment value.

5.2.3. Model Development

Once the problem has been defined, modelers must start generating assumptions as well as defining the standard values that define the models. These provide the basis for the model behavior and how the research question arose. First, assumptions are defined:
  • Distribution of finished products as well as the procurement of raw material is given;
  • Each order has a production unit;
  • Bill of materials (BOM) is not considered.
Moreover, the following items are considered to enable the comparison between the simulation models:
  • Same demand, use of demand replicas;
  • Same number of employees and capabilities;
  • Same production logic for all simulation models;
  • The warehouses without stock capacity limitations;
  • Transport between the different production stages has no limitations;
  • There are two products. One existing product is in its mature stage and has a stable demand and with a price of USD 10,000/unit. The new model is in the process of being launched with a price of USD 20,000/unit. These values were used to calculate sales. If there is loss in volume, it is assumed that the new product in development will have the loss in volume due to unknown future demand;
  • The simulation model considers a volume loss when the customer order lead time is greater than 60 days;
  • Time restrictions: 10 working years, i.e., 500 weeks, as the model attempts to evaluate the impacts in the medium and long term;

5.3. Design of Case Study Based on the DIIS

For the case study, databases from three companies are available. These databases serve as a basis for planning innovation projects of similar innovation types, such as modularization. The other three companies intend to introduce the innovation; for that purpose, the historical databases from the first three companies are used for initial planning. The three companies intending to introduce the innovation have a digital twin model of their supply chains and the related environment. By simulating different planning methodologies with the digital twin models and applying quality control charts for the key performance indicators, the DIIS applying the PDCA methodology is applied during the life cycle of innovation.

5.4. Development of Innovation Project Planning and Monitoring Based on the PDCA Methodology

An innovation project can be planned with different approaches depending on how to apply or not apply the PDCA methodology. As shown in Figure 10, the first methodology is standard planning, i.e., applying the same innovation introduction strategy to all companies and production plants, named “One fits all” planning. The second methodology is the static Do-Check planning methodology, which is based on the historical data and information from similar past innovation types, i.e., the initial definition of the innovation characteristics towards the target goals will be maintained until its introduction. The third methodology is dynamic Plan-Act planning, i.e., continuous or discrete (in milestones), for adjusting the innovation project after each Do-Check cycle. Although the first planning is based on the same historical data and information as the static Do-Check, the subsequent innovation plans and characteristics will be adapted if the modularization level is adjusted after each PDCA cycle. Moreover, for each of the planning methodologies, different assumptions and information are needed, as shown in Figure 10.

5.5. Validation of the DIIS

The validation of the simulation models can be carried out using different methods. In this process, some simulation variables will be used to observe their behavior and to evaluate whether the models will be validated. Sterman defined 12 possible methods to validate system dynamics models [62]. One of them, the test of extreme values, is used to validate the simulation model that shows that the response of the model is plausible when taking extreme values for different input parameters. For all models, the same input and output variables are chosen to analyze and validate the models. These input variables are the total number of employees and the absenteeism rate.
  • For a lower number of employees and higher absenteeism rate, profits (million USD), return on investment (ROI), and real demand (# thous. products) should be lower and the operational costs (million USD) should be higher. As shown in Figure 11, the model behaves as expected. Green lines represent the results for 500 employees, red lines are for 250 employees, and blue lines are for 100 employees.

6. Results

The results are depicted for three companies, each of them compared for the three planning methodologies, standard “one fits all” planning, static planning based on historical Do-Check, and dynamic Plan-Act planning based on current and historical Do-Check. All three companies in all different methodologies have the same target goals with two priority goals and several other second priority goals. These goals are a minimum of 280,000 of real demand satisfied, quality rate suppliers with a minimum of 95%, an absenteeism rate lower than 12%, an investment level lower than USD 1000 million, a profits level lower than USD 700 million, and an ROI higher than 10%. From these goals, the absenteeism rate and profit indicators are the first targets to be met, followed by the second priority goals. Moreover, the “one fits all” planning model derives the 40% modularization rate based on non-goal-oriented planning, i.e., a push-innovation strategy. On the other hand, static planning is already a pull-innovation strategy oriented to target goals based on the analysis of databases of the three companies that tried to introduce similar modularization strategies. Based on statistical analysis, 80% of modularization is derived and maintained throughout the innovation life cycle. Later, the dynamic Plan-Act planning modifies the original 80% modularization level depending on the results obtained along the innovation life cycle towards achieving the target goals at their two priority levels. For that purpose, in Table 1, Table 2 and Table 3, it can be observed how the dynamic model changes the percentage of modularization in each PDCA cycle. All companies present three PDCA cycles. The first is based on historical data from other companies at the initial time (t = 0 weeks) and remains until week 120. Then, a Plan-Act is performed based on the Do-Check evolution of the company, as well as the similar evolution of the other three companies. Based on this analysis, the modularization rate is reviewed, and a new value is provided and maintained until week 260. Then, the same process is followed, adjusting the modularization rate until week 500, the end of the innovation life cycle.
The results for the first company, as presented in Table 1, show how the best results in terms of production are obtained for the static historical Do-Check model, while the dynamic Plan-Act model is closer to the absenteeism rate of 12% as well as to the profit level of 700 million USD. Although optimization can be observed from the “one fits all” planning and the static planning, the dynamic Plan-Act model is not able to meet the priority goals at the same time with three iterative PDCA improvements. For this company, it needs to be assessed whether the introduction of the innovation makes sense considering the impacts on the overall indicators. On the one hand, the absenteeism rate improves slightly; however, profits decrease, and the ROI of the innovation is negative. Here, it is where the DIIS gains relevance and significance as it can predict these results, enabling any manager and decision-maker to assess the improvements if they are worth the investment and the negative results in some indicators.
The results for the second company, as presented in Table 2, show how the best results in terms of real demand are obtained for the dynamic Plan-Act model with almost 20,000 units more than the “One fits all” model. Moreover, the absenteeism rate is 9.5%, being lower than the target goal and 3% lower than the “One fits all” model. Furthermore, the profits level is above the goal of USD 700 million for all three planning methods while securing more than USD 150 million more for the Plan-Act model.
In addition, the second priority goals are also met with the Plan-Act model: the real demand is almost 10,000 units more than the goal of 280,000; the quality rate of suppliers is 96%, i.e., 1% more than the target value; the investment level is lower than 1000 million USD; and the ROI is higher than the 10% target value, improving from 26% to 37% from the static Do-Check to the dynamic Plan-Act system, respectively. In conclusion, it can be seen how the DIIS applied for the case study in this company not only enables the optimization of the achievement of target goals with different priority levels but also helps to balance the indicators during the innovation life cycle. The modularization level for this company is 80% until week 120 and 90% until week 260, with a final modularization rate of 85%. Thus, the innovation introduction is optimized in effectiveness, and its efficiency is focused on the priority indicators.
The results for the third company, as presented in Table 3, show how the best results in terms of real demand are obtained for the dynamic Plan-Act model with at least 20,000 units more than the “One fits all” model. Moreover, the absenteeism rate is 8.1%, being lower than the target goal and more than 3% lower than the “One fits all” model. Furthermore, the profits level is almost double the goal of USD 700 million while securing more than USD 200 million more for the dynamic Plan-Act model.
In addition, the second priority goals are also met with the Plan-Act model: the real demand is almost 20,000 units more than the goal of 280,000, and the quality rate of suppliers is higher than 97%, i.e., 2% more than the target value, the investment level is optimized and lower than USD 1000 million, and the ROI is six times higher than the 10% target value. In conclusion, it can be seen how the DIIS applied for the case study in this company enables the optimization of the achievement of target goals with different priority levels and supports the balance of indicators during the innovation life cycle. The modularization level for this company is 80% until week 120 and 85% until week 260, with a final modularization rate of 82.5%.
It is also relevant to mention how the results vary depending on the organization in which the innovation is introduced, although the innovation has the same technical characteristics. However, the companies have different capabilities and functions, thus providing different output results for the same input innovation. As a result, the importance of the DIIS for innovation project design, assessment, management, and monitoring is set.

7. Discussion

Firstly, the literature identified a need for an innovation management model that could cope with the environment’s complexity and define the gap in the existing application of innovation management tools, techniques, and systems for innovation. The conceptual model and the case study results show how a digital ecosystem can be developed and provide a platform for selecting, managing, and monitoring innovation projects based on the PDCA methodology. In addition, many companies perform assessments, especially at the conception of new ideas, inventions, and concepts, without introducing any information into a system, thus reducing the traceability and the learning options for the innovation project as well as for future innovation projects. In this regard, the DE4.0 and the DIIS provide a suitable response to this challenge, providing transparency and adaptability to enhance and develop dynamic capabilities, thus securing organizational sustainability.
Secondly, it is clear from the results that the organizational topology in terms of how a company is able to introduce an innovation is of vital importance. In some cases, this can be helpful when deciding whether to proceed or not with a certain innovation. This led us to the need to develop customized innovation plans depending on organizational topology. This topology is to be defined by the maturity level in a set of criteria such as culture, organization, technical, etc.
Thirdly, the PDCA methodology can be performed continuously or in discrete intervals. If the ecosystem is digital, it can be assessed at any time from any location, making it possible to perform adjustments in real time while others are performed at time intervals to evaluate indicator dynamics leading both options to efficient project management.
Fourthly, as shown from the results of the different companies, it has been described how the data, information, and learning from similar innovation project experiences are key when designing and deciding on new innovations. Therefore, this article suggests including the iterative learning process in the innovation life cycle as it provides significant advantages for the scope and scalability of the innovation introduction, as well as for its market diffusion.
Fifthly, the use of information systems connected with digital twins of an organization and its processes provides the basic framework for enhancing the system with simulation capabilities. These are key for identifying the policies and innovation characteristics that reduce the overall risks in the short and long term, thus not only for the future of the organization based on the innovation implementation but also for the sustainability of the whole company, its supply chain, and the related environment. With multiple sources of storing, treating, and sending/receiving data and information, cybersecurity plays a key role in these digital ecosystems. Furthermore, artificial intelligence is fundamental to enhancing the capabilities of the DE4.0 and the DIIS.
The advantage of this approach is the ability to convert all historical data, facts, and information from stakeholders and organizations, each with a specific set of capabilities, operating in different related market environments, into a fact- and scientific-based innovation plan. Moreover, this approach can improve dynamically over time thanks to the combination of elements that lead to a new management era that provides adaptability mechanisms oriented to organizational goals. Management is about running and changing any kind of organization. In this context, this research provides a step forward as it generates a new management framework based on innovation management according to current technological possibilities that enable ubiquitous information interconnection in a single source of truth. As a result, the paper recommends putting innovation at the core of organizational sustainability as the driver of change while securing future operating stability. Moreover, this management approach can enable flatter organizational structures based on the DE4.0 as well as the DIIS.
Although the existing possibilities for the development of a management era based on the fourth and in the advent of the fifth industrial revolution, decision-making in all planning horizons within and between organizations of any type is largely driven by pressure groups and based on subjective reasons while the real facts are, on many occasions, hidden. Many can argue that this is something of the past and that the advent of information technology and its development toward a digital world and knowledge economy have changed it already, but what happens after a plan or evaluation has been carefully made? Is it not changed or influenced by multiple non-controllable factors? The practice speaks for itself. Plans based on facts are made; however, they cannot change at the speed at which humans and their interactions affect the process. Many have also forgotten that new technologies and digital systems that are not based on real facts and data are useless. Research shows how the human factor in its interactions and decisions significantly affects the overall output of the organizational system [52,54,55,56]. Today, when top managers face a decision, typically, it is based on a set of slides prepared according to various agreement loops of multiple departments. In this context, the process up until the decision-making moment reduces the objectivity of data and facts, while an increase in subjectivity arises based on the person, department, and/or pressure-group-driven interests. As a result, top managers are forced to make decisions the same way a person takes a step in the direction of a single light in a dark room, suggesting technically viable and economically preferable solutions but missing the bigger picture based on facts, data, risk analysis, and their evaluation based on historical lessons learned. This is where the article provides a significant contribution as it mixes the real and digital world for a common beneficial purpose by providing transparency as the basis for better fact-based decision-making adaptable to real needs and customized for employees and managers to put the real facts and general interest first instead of person, department, or group-driven arguments and/or motivations.
Moreover, it is important to point out other aspects related to the persons involved in the innovation project when considering the factors affecting innovation success and reducing the risks for the company, supply-chain partners, and the related environment of the firm:
  • Screening of innovation team members: in the innovation project selection, a relevant factor is the members of the team that will work on the innovation. For this, screening potential candidates is a key measure for managers.
  • Creation of innovation teams: the selected members and their compatibility could be important factor managers and successful innovation results.
  • Development of high-performance teams: team members can, if managers decide, have direct contact with mentors or scientists supporting them in the innovation process and its management.
  • Innovation and technology evaluators: top managers must decide how to select an evaluator or group of evaluators for the assessment. Moreover, it could be the case that different evaluators assessed the innovation in different stages of the innovation journey. Furthermore, managers should specify if there is a guide of good practice for evaluators.
  • Synergy creation: the presentation of the innovation progress and how it is managed to other innovation teams could help to generate synergies for existing innovation projects as well as generate new ideas for future innovations.

8. Conclusions

Based on the research performed, it was concluded that there is a need to study how innovation projects are decided, monitored, and improved along the innovation life cycle within a company. On the one hand, an innovation model was generated applying the Viable System Model that ensures that new initiatives are aligned with organizational goals and adapted to the environment dynamics. On the other hand, we developed a project evaluation scheme to be applied with a Digital Ecosystem for the Fourth Industrial Revolution (DE4.0) in the Dynamic Innovation Information System (DIIS) that serves as a basis to assess innovation projects dynamically, by knowing the status of organizational functionalities or capabilities in each moment with the help of a digital twin model and by using simulation means.
Furthermore, the paper provided a combination of conceptual modeling with the Viable System Model, leading to a structure for different planning horizons, strategies, tactics, and operations for environmental and market dynamics. It also presented a novel methodology for the development of digital ecosystems in the Fourth Industrial Revolution consisting of information systems and their related databases, digital twin models of management and technical processes and assets, simulation capabilities, and models while applying quality management and control techniques as well as statistical analysis. This set of elements developed a unique methodology, DE4.0, that can enhance the effectiveness and efficiency of any information ecosystem. Therefore, it was applied to innovation management and innovation project management in the DIIS based on the PDCA methodology.
Moreover, this research provides managers with the decision-making capabilities that they need to consider when deciding which innovations must be developed, how they should be developed, and which impacts, positive and negative, they will have to enlarge the positive effects on the value chain and develop and plan preventive measures to avoid or minimize the negative effects. Therefore, the DIIS based on the PDCA methodology, and the results developed in the paper show that managers need to consider this digital ecosystem, as it increases the probability of reaching their targets. For this purpose, organizational strategic goals, supply chain areas, new technologies, and new product characteristics need to be known. Based on this step, managers are provided with a model in which they can orientate their decision-making activities, with a focus on the organizational targets in each PDCA cycle.
The limitations of this research are as follows:
  • Generic case study model, which needs specific sector knowledge for its application;
  • Digital twin model is not based on a real supply chain;
  • Product characteristics are not specified for particular use cases;
  • Limited ability of the model to capture qualitative information in certain events and moments of past and current innovation plans and projects.
Future research could focus on the following:
  • Apply in real organizations;
  • Implement different new products and different technologies from different sectors;
  • Implement all areas of the conceptual model in detail in a digital twin model with simulation and artificial intelligence capabilities.
To sum up, the research shows the potential benefits of the conceptual model for decision-making in innovation projects. As a result, the proposed methodology provides a useful conceptual model, a novel digital ecosystem (DE4.0), and a Dynamic Innovation Information System (DIIS) based on the PDCA methodology applicable for any organization when deciding and managing their innovation project portfolio thanks to the dynamic evaluation of impacts.

Author Contributions

Conceptualization, S.G.-G. and M.G.-G.; Methodology, S.G.-G.; Software and programming, S.G.-G.; Validation, S.G.-G.; Data Analysis, S.G.-G., D.G.-G., D.R. and S.P.-G.; Writing (review and editing), S.G.-G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Eveleens, C. Innovation management; a literature review of innovation process models and their implications. Science 2010, 800, 900. [Google Scholar]
  2. Hidalgo, A.; Albors, J. Innovation management techniques and tools: A review from theory and practice. R&D Manag. 2008, 38, 113–127. [Google Scholar] [CrossRef] [Green Version]
  3. Lopes, A.; Polónia, D.; Gradim, A.; Cunha, J. Challenges in the Integration of Quality and Innovation Management Systems. Standards 2022, 2, 52–65. [Google Scholar] [CrossRef]
  4. Marinova, D.; Phillimore, J. Models of innovation. In The International Handbook on Innovation; Elsevier: Amsterdam, The Netherlands, 2003; Volume 1. [Google Scholar]
  5. Martensen, A.; Dahlgaard, J.J. Strategy and planning for innovation management—A business excellence approach. Int. J. Qual. Reliab. Manag. 1999, 16, 734–755. [Google Scholar] [CrossRef]
  6. Ettlie, J.E.; Bridges, W.P.; O’Keefe, R.D. Organization Strategy and Structural Differences for Radical Versus Incremental Innovation. Manag. Sci. 1984, 30, 682–695. [Google Scholar] [CrossRef]
  7. Kapsali, M. Systems thinking in innovation project management: A match that works. Int. J. Proj. Manag. 2011, 29, 396–407. [Google Scholar] [CrossRef]
  8. Kock, A.; Georg Gemünden, H. Antecedents to decision-making quality and agility in innovation portfolio man-agement. J. Prod. Innov. Manag. 2016, 33, 670–686. [Google Scholar] [CrossRef]
  9. Schuh, G. (Ed.) Innovationsmanagement: Handbuch Produktion und Management 3; Springer: Berlin/Heidelberg, Germany, 2012. [Google Scholar]
  10. Žižlavský, O. Net present value approach: Method for economic assessment of innovation projects. Procedia Soc. Behav. Sci. 2014, 156, 506–512. [Google Scholar] [CrossRef]
  11. Tomala, F.; Sénéchal, O. Innovation management: A synthesis of academic and industrial points of view. Int. J. Proj. Manag. 2004, 22, 281–287. [Google Scholar] [CrossRef]
  12. Consejo Superior de Investigaciones Científicas. La Investigación en la Universidad Española: La Financiación Competitiva de la Investigación, con Especial Referencia a las Ciencias Sociales y Económicas; Sanz, CSIC: Madrid, Spain, 2003.
  13. Gobierno de España. Estrategia Española de Ciencia y Tecnología y de Innovación 2013–2020; Ministerio de Economía y Competitividad, Gobierno de España: Madrid, Spain, 2013.
  14. Hnatenko, I.; Orlova-Kurilova, O.; Shtuler, I.; Serzhanov, V.; Rubezhanska, V. An approach to innovation potential evaluation as a means of enterprise. Int. J. Supply Oper. Manag. 2020, 7, 112–118. [Google Scholar]
  15. Trott, P. Innovation Management and New Product Development; Financial Times Prentice Hall: Harlow, UK, 2005. [Google Scholar]
  16. Brook, J.W.; Pagnanelli, F. Integrating sustainability into innovation project portfolio management—A strategic perspective. J. Eng. Technol. Manag. 2014, 34, 46–62. [Google Scholar] [CrossRef]
  17. Mytelka, L.K.; Smith, K.; Karelplein, K. Innovation theory and innovation policy: Bridging the gap. In Social Sciences and Innovation; OECD: Paris, France, 2001; p. 125. [Google Scholar]
  18. Foss, N.J.; Saebi, T. Fifteen years of research on business model innovation: How far have we come, and where should we go? J. Manag. 2017, 43, 200–227. [Google Scholar] [CrossRef] [Green Version]
  19. Havlíček, K.; Thalassinos, E.; Berezkinova, L. Innovation Management and Controlling in SMEs. Eur. Res. Stud. J. 2013, XVI, 57–70. [Google Scholar] [CrossRef] [Green Version]
  20. Adegbile, A.; Sarpong, D.; Meissner, D. Strategic Foresight for Innovation Management: A Review and Research Agenda. Int. J. Innov. Technol. Manag. 2017, 14, 1750019. [Google Scholar] [CrossRef]
  21. Flamini, G.; Pellegrini, M.M.; Manesh, M.F.; Caputo, A. Entrepreneurial approach for open innovation: Opening new opportunities, mapping knowledge and highlighting gaps. Int. J. Entrep. Behav. Res. 2021. [Google Scholar] [CrossRef]
  22. Espejo, R.; Harnden, R. (Eds.) The Viable System Model: Interpretations and Applications of Stafford Beer’s VSM; Wiley: Hoboken, NJ, USA, 1989. [Google Scholar]
  23. Autiosalo, J. Platform for industrial internet and digital twin focused education, research, and innovation: Ilmatar the overhead crane. In Proceedings of the 2018 IEEE 4th World Forum on Internet of Things (WF-IoT), Singapore, 5–8 February 2018; pp. 241–244. [Google Scholar]
  24. Lim, K.Y.H.; Zheng, P.; Chen, C.-H. A state-of-the-art survey of Digital Twin: Techniques, engineering product lifecycle management and business innovation perspectives. J. Intell. Manuf. 2020, 31, 1313–1337. [Google Scholar] [CrossRef]
  25. He, B.; Bai, K.-J. Digital twin-based sustainable intelligent manufacturing: A review. Adv. Manuf. 2021, 9, 1–21. [Google Scholar] [CrossRef]
  26. Albors-Garrigos, J.; Igartua, J.I.; Peiro, A. Innovation Management Techniques and Tools: Its Impact on Firm Innovation Performance. Int. J. Innov. Manag. 2018, 22, 1850051. [Google Scholar] [CrossRef]
  27. Lipczynski, J.; Wilson, J.O.; Goddard, J.A. Industrial Organization: Competition, Strategy, Policy; Pearson Education: New York, NY, USA, 2005. [Google Scholar]
  28. Schuh, G.; Weber, H.; Kajüter, P. Logistikmanagement; Springer: Berlin/Heidelberg, Germany, 2013; pp. 22–23. [Google Scholar]
  29. Tidd, J.; Bessant, J. Innovation Management Challenges: From Fads to Fundamentals. Int. J. Innov. Manag. 2018, 22, 1840007. [Google Scholar] [CrossRef]
  30. Rothwell, R. Towards the Fifth-generation Innovation Process. Int. Mark. Rev. 1994, 11, 7–31. [Google Scholar] [CrossRef]
  31. Nechaev, A.S.; Ognev, D.V.; Antipina, O.V. Analysis of risk management in innovation activity process. In Proceedings of the 2017 International Conference “Quality Management, Transport and Information Security, Information Technologies”(IT&QM&IS), IEEE, St. Petersburg, Russia, 24–30 September 2017; pp. 548–551. [Google Scholar]
  32. Beer, S.; Beer, T. The Heart of Enterprise; Wiley: Chichester, UK; New York, NY, USA, 1979; Volume 2. [Google Scholar]
  33. Beer, S. Brain of the Firm: A Development in Management Cybernetics Herder and Herder; Verlag Herder: Freiburg im Breisgau, Germany, 1972. [Google Scholar]
  34. Espejo, R.; Schuhmann, W.; Schwaninger, M.; Bilello, U. Organizational Transformation and Learning: A Cybernetic Approach to Management; Wiley: Hoboken, NJ, USA, 1996; p. 350. [Google Scholar]
  35. Oztemel, E.; Gursev, S. Literature review of Industry 4.0 and related technologies. J. Intell. Manuf. 2020, 31, 127–182. [Google Scholar] [CrossRef]
  36. Tao, F.; Qi, Q. New IT Driven Service-Oriented Smart Manufacturing: Framework and Characteristics. IEEE Trans. Syst. Man Cybern. Syst. 2017, 49, 81–91. [Google Scholar] [CrossRef]
  37. Stock, T.; Seliger, G. Opportunities of sustainable manufacturing in industry 4.0. Procedia CIRP 2016, 40, 536–541. [Google Scholar] [CrossRef] [Green Version]
  38. Wang, S.; Wan, J.; Li, D.; Zhang, C. Implementing Smart Factory of Industrie 4.0: An Outlook. Int. J. Distrib. Sens. Netw. 2016, 12, 3159805. [Google Scholar] [CrossRef] [Green Version]
  39. Schuh, G.; Stich, V.; Brosze, T.; Fuchs, S.; Pulz, C.; Quick, J.; Schürmeyer, M.; Bauhoff, F. High resolution supply chain management: Optimized processes based on self-optimizing control loops and real time data. Prod. Eng. 2011, 5, 433–442. [Google Scholar] [CrossRef]
  40. Herold, C. Ein Vorgehenskonzept zur Unternehmensstrukturierung Heuristische Anwengung des Modells Lebensfahiger Systeme; Difo Druck: Bamberg, Germany, 1991. [Google Scholar]
  41. Angerhofer, B.; Angelides, M. System dynamics modelling in supply chain management: Research review. In Proceedings of the 2000 Winter Simulation Conference Proceedings (Cat. No. 00CH37165), Orlando, FL, USA, 10–13 December 2000; pp. 342–351. [Google Scholar]
  42. Sterman, J.D. Business Dynamics: Systems Thinking and Modeling for a Complex World; Irwin/McGraw-Hill: New York, NY, USA, 2000. [Google Scholar]
  43. Reggelin, T. Schneller Entscheiden. Log. Kompass; DVV Media Group: Hamburg, Germany, 2012; p. 5. [Google Scholar]
  44. Gallego García, S.; García García, M. Design and Simulation of Production and Maintenance Management Applying the Viable System Model: The Case of an OEM Plant. Materials 2018, 11, 1346. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  45. Gallego-García, S. Design and Simulation of a Logistics Distribution Network through Applying the Viable System Model (VSM). Master’s Thesis and Final Degree Project, RWTH, Aachen, Germany, University of Saragossa, Zaragoza, Spain, 2015. [Google Scholar]
  46. Reyes, R.R.; García, S.G.; García, M.G. Applying the Viable System Model to an Organization with CSR Goals: The Case of a Charity Organization. Eur. J. Econ. Bus. Stud. 2018, 4, 7–21. [Google Scholar] [CrossRef]
  47. Gallego-García, S.; García-García, M. Market-oriented procurement planning leading to a higher service level and cost optimization. Appl. Sci. 2020, 10, 8734. [Google Scholar] [CrossRef]
  48. Gejo-García, J.; Gallego-García, D.; Gallego-García, S.; García-García, M. Simulation Model of a Spare Parts Distribution Network in the Airline Industry for Reducing Delays and Improving Service Levels: A Design of Experiments Study. In International Conference on Intelligent Human Computer Interaction; Springer: Cham, Switzerland, 2021; pp. 745–760. [Google Scholar]
  49. Gallego-García, S.; Gejo-García, J.; García-García, M. Development of a maintenance and spare parts distribution model for increasing aircraft efficiency. Appl. Sci. 2021, 11, 1333. [Google Scholar] [CrossRef]
  50. Gejo-García, J.; Reschke, J.; Gallego-García, S.; García-García, M. Development of a System Dynamics Simulation for Assessing Manufacturing Systems Based on the Digital Twin Concept. Appl. Sci. 2022, 12, 2095. [Google Scholar] [CrossRef]
  51. Gallego-García, S.; García-García, M. Design and Simulation of an Integrated Model for Organisational Sustainability Applying the Viable System Model and System Dynamics. In IFIP International Conference on Advances in Production Management Systems; Springer: Cham, Switzerland, 2019; pp. 555–563. [Google Scholar]
  52. Gallego-García, D.; Gallego-García, S.; García-García, M. An Optimized System to Reduce Procurement Risks and Stock-Outs: A Simulation Case Study for a Component Manufacturer. Appl. Sci. 2021, 11, 10374. [Google Scholar] [CrossRef]
  53. Gallego-García, S.; García-García, M. Predictive sales and operations planning based on a statistical treatment of demand to increase efficiency: A supply chain simulation case study. Appl. Sci. 2020, 11, 233. [Google Scholar] [CrossRef]
  54. Torres, M.B.; Gallego-García, D.; Gallego-García, S.; García-García, M. Development of a Business Assessment and Diagnosis Tool That Considers the Impact of the Human Factor during Industrial Revolutions. Sustainability 2022, 14, 940. [Google Scholar] [CrossRef]
  55. Ren, D.; Gallego-García, D.; Pérez-García, S.; Gallego-García, S.; García-García, M. Modeling Human Decision-Making Delays and Their Impacts on Supply Chain System Performance: A Case Study. In International Conference on Intelligent Human Computer Interaction; Springer: Cham, Switzerland, 2021; pp. 673–688. [Google Scholar]
  56. Winkler, M.; Gallego-García, S.; García-García, M. Design and Simulation of Manufacturing Organizations Based on a Novel Function-Based Concept. Appl. Sci. 2022, 12, 811. [Google Scholar] [CrossRef]
  57. Winkler, P.; Gallego-García, S.; Groten, M. Design and Simulation of a Digital Twin Mobility Concept: An Electric Aviation System Dynamics Case Study with Capacity Constraints. Appl. Sci. 2022, 12, 848. [Google Scholar] [CrossRef]
  58. Gejo Garcia, J.; Gallego-Garcia, S.; Garcia-Garcia, M. Development of a pull production control method for ETO companies and simulation for the metallurgical industry. Appl. Sci. 2019, 10, 274. [Google Scholar] [CrossRef] [Green Version]
  59. Gallego-García, S.; Groten, M.; Halstrick, J. Integration of Improvement Strategies and Industry 4.0 Technologies in a Dynamic Evaluation Model for Target-Oriented Optimization. Appl. Sci. 2022, 12, 1530. [Google Scholar] [CrossRef]
  60. Gallego-García, S.; Reschke, J.; García-García, M. Design and simulation of a capacity management model using a digital twin approach based on the viable system model: Case study of an automotive plant. Appl. Sci. 2019, 9, 5567. [Google Scholar] [CrossRef] [Green Version]
  61. Groten, M.; Gallego-García, S. A Systematic Improvement Model to Optimize Production Systems within Industry 4.0 Environments: A Simulation Case Study. Appl. Sci. 2021, 11, 11112. [Google Scholar] [CrossRef]
  62. Wensing, T. Periodic Review Inventory Systems. In Performance Analysis and Optimization of Inventory Systems within Supply Chains; Springer: Berlin/Heidelberg, Germany, 2011. [Google Scholar]
Figure 1. Innovation management tasks according to planning horizons (own elaboration based on [44]).
Figure 1. Innovation management tasks according to planning horizons (own elaboration based on [44]).
Applsci 12 06592 g001
Figure 2. VSM recursion levels for company, innovation, technical process (own elaboration based on [44,45]).
Figure 2. VSM recursion levels for company, innovation, technical process (own elaboration based on [44,45]).
Applsci 12 06592 g002
Figure 3. VSM recursion level for the company (own elaboration based on [44,45,46].
Figure 3. VSM recursion level for the company (own elaboration based on [44,45,46].
Applsci 12 06592 g003
Figure 4. VSM analogy with innovation management (own elaboration).
Figure 4. VSM analogy with innovation management (own elaboration).
Applsci 12 06592 g004
Figure 5. Project evaluation: three levels and 11 steps in an iterative process (own elaboration).
Figure 5. Project evaluation: three levels and 11 steps in an iterative process (own elaboration).
Applsci 12 06592 g005
Figure 6. A Digital Ecosystem for the Fourth Industrial Revolution (DE4.0) (own elaboration).
Figure 6. A Digital Ecosystem for the Fourth Industrial Revolution (DE4.0) (own elaboration).
Applsci 12 06592 g006
Figure 7. The Dynamic Information Innovation System (DIIS) (own elaboration).
Figure 7. The Dynamic Information Innovation System (DIIS) (own elaboration).
Applsci 12 06592 g007
Figure 8. Innovation life cycle and project planning based on the PDCA methodology (own elaboration).
Figure 8. Innovation life cycle and project planning based on the PDCA methodology (own elaboration).
Applsci 12 06592 g008
Figure 9. Structure of the simulation case study [61].
Figure 9. Structure of the simulation case study [61].
Applsci 12 06592 g009
Figure 10. Innovation Project Planning Methodologies based on the PDCA approach (own elaboration).
Figure 10. Innovation Project Planning Methodologies based on the PDCA approach (own elaboration).
Applsci 12 06592 g010
Figure 11. Validation based on extreme-value tests for number of employees (own elaboration).
Figure 11. Validation based on extreme-value tests for number of employees (own elaboration).
Applsci 12 06592 g011
Table 1. Simulation results for the Innovation Case Study: Product Modularization in Company 1.
Table 1. Simulation results for the Innovation Case Study: Product Modularization in Company 1.
No.Key IndicatorCompany 1
“One Fits All”
Planning
Static Planning Based on Historical Do-CheckDynamic Plan-Act Planning Based on Current and Historical Do-Check
40% Modularization80% Modularization80% Mod.: 0–120 Weeks
60% Mod.: 120–260 Weeks
65% Mod.: 260–500 Weeks
1∑ Demand (# 103 products)308,738308,738308,738
2∑ Demand real (#103 products)264,087276,187270,730
3∑ Production (# products)234,366247,585241,552
4Ø Plant availability rate (%)83.484.884.3
5Ø Performance rate (%)91.7101.597.8
6Development time (weeks)489678
7Absenteeism rate (%)13.311.412.1
8Labor productivity (units/empl. × day)46.949.548.3
9Ø Production lead time (# days)98.489.893.2
10Ø Suppliers’ quality rate (%)92.094.093.3
11Cumulated service level (%)95.296.095.8
12∑ Sales (million USD)387841204010
13∑ Operational costs (million USD)265525272594
14∑ Investment (million USD)5181046803
15∑ Profits (million USD)705547614
16Return on investment (ROI) (%)−0.5−15.3−11.6
Table 2. Simulation results for the Innovation Case Study: Product Modularization in Company 2.
Table 2. Simulation results for the Innovation Case Study: Product Modularization in Company 2.
No.Key IndicatorCompany 2
“One Fits All”
Planning
Static Planning Based on Historical Do-CheckDynamic Plan-Act Planning Based on Current and Historical Do-Check
40% Modularization80% Modularization80% Mod.: 0–120 Weeks
90% Mod.: 120–260 Weeks
85% Mod.: 260–500 Weeks
1∑ Demand (# 103 products)308,738308,738308,738
2∑ Demand real (#103 products)270,267286,840289,848
3∑ Production (# products)240,967259,900263,442
4Ø Plant availability rate (%)84.186.186.5
5Ø Performance rate (%)96.0105.8107.6
6Development time (weeks)4896102
7Absenteeism rate (%)12.59.89.5
8Labor productivity (units/empl. × day)48.252.052.7
9Ø Production lead time (# days)94.982.380.4
10Ø Suppliers’ quality rate (%)92.895.696.0
11Cumulated service level (%)95.896.296.4
12∑ Sales (million USD)400143334393
13∑ Operational costs (million USD)257523902347
14∑ Investment (million USD)499979976
15∑ Profits (million USD)9279641070
16Return on investment (ROI) (%)44.026.237.2
Table 3. Simulation results for the Innovation Case Study: Product Modularization in Company 3.
Table 3. Simulation results for the Innovation Case Study: Product Modularization in Company 3.
No.Key IndicatorCompany 3
“One Fits All”
Planning
Static Planning Based on Historical Do-CheckDynamic Plan-Act Planning Based on Current and Historical Do-Check
40% Modularization80% Modularization80% Mod.: 0–120 Weeks
85% Mod.: 120–260 Weeks
82.5% Mod.: 260–500 Weeks
1∑ Demand (# 103 products)308,738308,738308,738
2∑ Demand real (#103 products)276,347297,114298,955
3∑ Production (# products)247,639272,573241,552
4Ø Plant availability rate (%)84.987.487.7
5Ø Performance rate (%)99.5112.2113.3
6Development time (weeks)489699
7Absenteeism rate (%)11.78.38.1
8Labor productivity (units/empl. × day)49.554.555.0
9Ø Production lead time (# days)91.274.172.8
10Ø Suppliers’ quality rate (%)93.697.297.4
11Cumulated service level (%)96.096.696.8
12∑ Sales (million USD)412345384575
13∑ Operational costs (million USD)249422482220
14∑ Investment (million USD)484970997
15∑ Profits (million USD)114413201358
16Return on investment (ROI) (%)90.263.365.3
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Gallego-García, S.; Ren, D.; Gallego-García, D.; Pérez-García, S.; García-García, M. Dynamic Innovation Information System (DIIS) for a New Management Age. Appl. Sci. 2022, 12, 6592. https://doi.org/10.3390/app12136592

AMA Style

Gallego-García S, Ren D, Gallego-García D, Pérez-García S, García-García M. Dynamic Innovation Information System (DIIS) for a New Management Age. Applied Sciences. 2022; 12(13):6592. https://doi.org/10.3390/app12136592

Chicago/Turabian Style

Gallego-García, Sergio, Diqian Ren, Diego Gallego-García, Salvador Pérez-García, and Manuel García-García. 2022. "Dynamic Innovation Information System (DIIS) for a New Management Age" Applied Sciences 12, no. 13: 6592. https://doi.org/10.3390/app12136592

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop