Interorganizational Collaboration in Innovation Networks: An Agent Based Model for Responsible Research and Innovation in Additive Manufacturing

Abstract

The significant progress in scientific research and innovation has led to the need for a new paradigm to legitimise the innovation process in society and politics. The European Union, with the Horizon 2020 framework program and Horizon Europe, institutionalises this change by defining the concept of responsible research and innovation (RRI), aiming at greater inclusiveness and sustainability in the research and innovation processes. This paper aimed to present an agent-based model (ABM) to simulate the dynamics between the different actors that cooperate within networks during the innovation process, taking the inclinations toward RRI practices into account. The different types of agent, their characteristics, and the different strategies that they follow have been formulated within the Horizon 2020 project I AM RRI-Webs of Innovation Value Chains (IVCs) of Additive Manufacturing (AM) under consideration of RRI. Besides, some experiments are reported to validate the model, ensuring its rigor and making our model a useful tool for policymakers, assisting them in defining strategic guidelines for disseminating and encouraging RRI best practices and defining the critical factors of the innovative cooperative process.

1. Introduction

In the last few years, the progress in scientific research and innovation has led to an increase in relevant ethical, legal, and social issues concerning the relationship between science and society. Only an intensive dialogue between science and society ensures more responsible scientific research and innovation, as well as broader social and political support for research. Indeed, public investment in science requires a social and political base as broad as possible that shares its values and recognise its contributions to knowledge and economic progress.

It is in this spirit that the European Union, with the Horizon 2020 [1] framework program, and with its successor, Horizon Europe [2], inaugurated this year (2021), aimed for a new approach that anticipates and assesses the potential implications and societal expectations about research and innovation, with the purpose of fostering the design and implementation of inclusive and sustainable research and innovation: responsible research and innovation (RRI) [3]. With this new vision, all social actors (researchers, citizens, policymakers, business, third sector organisations, and others) cooperate to align the innovation process and its outcomes with societal expectations and values.

RRI has not only inspired the objectives of Horizon 2020 but it has been the key action of the Horizon 2020 transversal program “Science with and for Society” (SwafS) [2].

One of the most critical challenges of the SwafS program is “to model and better understand the dynamics of the complex webs of innovation value chains and the openings they offer for RRI” [4] (p. 27), to which the project I AM RRI-Webs of Innovation Value Chains (IVCs) of Additive Manufacturing (AM) under consideration of RRI [5] responds. The aim is to develop a dynamic model to simulate the behaviour of a web of IVCs and their complex interactions under various conditions to arrive at an overall understanding of the opportunities provided by RRI practices. This will foster the adoption and dissemination of good RRI practices, also promoting institutional changes in research funding organisations (RFO) and research performing organisations (RPO).

The concept of responsible innovation, which later officially became RRI, has become very popular in Europe (and beyond) in recent years; this is also due to the Horizon 2020 project’s impetus, which is an essential and universal theme. The Rome Declaration on RRI provides a precise definition: RRI is an “on-going process of aligning research and innovation with the values, needs and expectations of society” [6] (p. 1).

This definition has two main points: the first is related to the idea that RRI is a dynamic concept, never the same, always growing. Each of its elements (accountability, research, and innovation) does not keep static [7]; all the elements, individually and all together, are in continuous movement. The second pivotal point is undoubtedly the need to align with society and its values, needs, and expectations; this is the goal of RRI, which is rooted in responsibility.

According to Van den Hove et al. [8], RRI goes beyond creating just economic growth, as it aims to benefit people by meeting their needs and by providing economic, environmental, and social sustainability. As a consequence, studying how RRI openings and practices enable valuable interactions among diverse business actors, research agents, citizens, and other stakeholders could be an interesting research topic to approach the sustainability of innovation from a multi-perspective point of view.

According to the purposes of the I AM RRI project, we adopted the agent-based modelling (ABM) approach as a focal methodology to investigate how RRI practices can impact on the interorganisational collaboration dynamics of innovation networks in AM.

In the last two decades, ABM has attracted considerable attention in different research areas, such as organisation studies [9,10], innovation networks [11,12,13], logistics and supply chain management [14,15,16,17,18], decision-making, economics, and financial networks [19,20,21,22,23,24,25,26]. ABM is particularly suited to studying complex adaptive systems [27,28,29]. In fact, despite the possible use of ABM as a traditional simulation methodology adopted for optimisation or to increase the generalisability of statistical analysis, this research methodology allows us to consider the autonomy and the heterogeneity of the components of a system and the non-linearity of the interactions among them and with the environment [30]. In the presence of high complexity, formal quantitative approaches require a reductionist intervention that results in an oversimplification of the model. ABM, by merging the flexibility of descriptive models and the possibility of calculus offered by computer implementations, supports researchers to account for both the individual and the systemic levels, the emerging collective behaviours in social systems, and self-organised paths developing from the bottom up [30,31].

Definitively, ABM fits well with the analysis of dynamic systems strongly characterised by the interactions and collaborations among heterogenous agents, and for exploring, through generative experiments [32], the collective properties (e.g., knowledge diffusion, organisational learning, and technological or social innovation) that are not linearly related to individual and local rules.

The developed ABM takes its cue for the modelling of agent knowledge and other endogenous characteristics from the existing ABM SKIN (Simulating Knowledge dynamics in Innovation Networks) [33,34] and enriches it with new elements and new adaptations by bringing it closer to empirical experience; hence it has been named I AM RRI SKIN.

The proposed model is a “double-industry” model in which agents, representing innovative firms, can belong to the automotive industry, the biomedical applications industry, or both (this is the case of “broker” agents, who are able to be involved in IVCs in both industries). The interconnection between the two industries is ensured by the presence of “broker” agents and by the fact that an agent can simultaneously participate in several networks (or IVCs), which belong to the automotive or biomedical industry, according to the characteristics of the agent initiating the network. In this way, the interconnection between various IVCs it is also modelled.

The model is based on a diversity of agents, representing different types of actor (AM technology companies, suppliers, customers, research institutions, and OEMs) assessed by regulators and standard organisations according to their RRI values and the quality of the innovation proposed. Heterogeneity affects diverse aspects of the agents modelled: they belong to different categories (business actors, customers or consumers, and research players), they are endowed with different capabilities, and they have different RRI inclinations. This diversity is one of the main drivers of collaborative relations in innovation networks, since agents need to find complementary competencies to effectively learn and develop an innovation project.

Furthermore, the I AM RRI SKIN model is a dynamic model in which the IVCs evolve in two sequential phases: idea generation and product development [35], each with a different duration and, at the end of which, a “gate” appears.

The agents’ inclination towards RRI values is modelled through three proxy variables (endogenous to the model), representing three fundamental thematic areas in RRI spreading as identified by the European Commission: public engagement, open-access, and ethical thinking. The agents’ decisions, the IVCs’ performances, and the way in which agents interact with the environment and with other actors depend on these RRI characteristics. In addition, the model is easily adaptable to other contexts where networking is needed and knowledge is exchanged among participants.

Therefore, the contribution of this paper is to present a simulative ABM investigating the role of RRI openings and heterogeneity of the agents’ knowledge in the formation and in the dynamics of collaborative and innovative relations among partners of innovation networks.

Finally, after presenting the general aspects of the model, some experiments will be illustrated to ensure the model’s rigor, observe its behaviour when the most interesting parameters change, test its implementation, and validate its agreement with the evidence in prior literature.

2. Materials and Methods

According to the literature about the development of computational models [36,37,38,39,40,41], the methodology used to build up the I AM RRI SKIN ABM can be articulated in 5 main steps, as depicted in Figure 1.

In the first step, the insights coming from the theoretical background analysed in the preliminary steps of the I AM RRI project (which are not the central topic of the paper) were translated into a conceptual model, reported and described as various deliverables, some of which will soon be available on the project web page (https://www.iamrri.eu/ (accessed on 27 April 2021)). For the development of our ABM, the theoretical background was also represented by the original SKIN model and its extensions.

The second step, which is the construction of the meta-model, helps in defining the blocks of the ABM and a pseudo-code in which micro-specifications about the agents’ behaviours are reported. In our case, the meta-model specified the agent classes, the rules of action and interactions among them, and also the variables to be considered in future simulations of the model. Our meta-model has been developed by starting from the conceptualisation of the SKIN model, but it has been enriched by a new interpretation of the original building blocks and also by new features devoted to increasing the representativeness of the model with respect to the system under investigation.

The implementation of the meta-model required the choice of an adequate software platform and the activity of code writing. The current version of the model, described here, has been implemented using NetLogo 6.1.1. (https://ccl.northwestern.edu/netlogo/ ( accessed on 27 April 2021)). The implemented model was subjected to an internal analysis aimed at verifying the consistency between the conceptual model and meta-model (Step 4); the methodology, experiments, and results of the validation phase are described in detail in Section 4.

Currently, the model is at the beginning of the validation phase, comparing the model’s behaviour to evidence in the literature. The project’s future developments will focus on deepening this last phase, using empirical data provided by the project’s industrial partners. Figure 1 schematises the methodology used.

3. The Model

The aim of the I AM RRI project is to develop a complex network model of AM innovation chains and their associated processes, which is directed towards RRI at all levels.

The developed model, coded in NetLogo using an ABM approach, originated as an extension of the already existing SKIN model [33,34,42,43], and it is mainly focused on the study of IVCs, webs of IVCs, and especially their openings for RRI. AM was taken as the basis for developing the model, given the characteristics of its innovation system, which are prone to the creation of webs of IVCs and are different from those of other traditional industries. AM is also an industry that has only recently started to evolve, following innovative approaches to manufacturing; it is already valued at more than $11 billion and growing [44]. However, the model is easily adaptable to other types of industries.

The I AM RRI SKIN model incorporates complexity, covering various stages of the IVC’s life, mainly idea generation and product development. The development of the innovation process goes through phases in which the capabilities (large domains of knowledge) and the abilities (applications in these domains) needed for an idea to be further developed are not well defined. Therefore, the cooperation between the agents and the creation of networks of IVCs are essential.

Moreover, unlike SKIN, from which I AM RRI SKIN was derived, the innovation process develops through different “ticks” (the simulation’s discrete time unit) and it is not obtained in just one model running cycle. In fact, in SKIN, a firm completes the whole process (from the search for partners to the sale of the products) in a single tick. Each IVC phase modelled covers a defined number of running cycles, defined as a result of the study conducted in the I AM RRI project.

There are mainly two implemented IVC phases, as mentioned above:

• Idea generation (3 ticks);
• Product development (12 ticks).

This choice derives from the study of use-cases (data concerning the use-cases, assumptions and results will be available as soon as possible on the webpage https://www.iamrri.eu (accessed on 27 April 2021)) provided by the partners of the I AM RRI project (reported in the Acknowledgments below) operating in the AM automotive or biomedical industry, which ensured realism in the model. The main time-related assumption made by the project is that 1 tick equals 1 month. Additionally, as the main difference between SKIN and I AM RRI SKIN, the decisions and the behaviour of the agents were not only price-related and cost-related, but time-related and especially RRI-related.

Each agent was equipped with three RRI state variables representing RRI inclinations that are translated into the model in the following three keys:

• Open access;
• Public engagement;
• Ethical thinking.

These keys profoundly influence the decision-making process.

Other relatively minor extensions have been introduced to adapt the I AM RRI SKIN model for automotive and biomedical use-cases. A double-industry model has been built, in which six different types of agent breeds—particular typologies of agent sets endowed with particular variables and characteristics that make them different from all other agent types—interact among each other and also participate in more networks simultaneously. This implies that the agents in the model can belong to the automotive industry, the biomedical applications industry, or both (this is the case of “broker” agents, who are able to be involved in IVCs in both industries). The inter-relationships between the two industries are ensured by “broker” agents and also by the fact that agents in the I AM RRI SKIN model can belong to more than one network at the same time. Identifications of the different types of agent, such as the duration of the IVC phases, came from the analysis of use-cases provided by the partners of the AM RRI, which cover two specialisations of the AM industry: automotive and biomedical.

The six breeds implemented were:

• AM technology companies;
• Suppliers;
• Customers;
• OEMs;
• Research institutions;
• Networks.

Other agents, such as the funding organisations, regulators, and the standard organisations, are modelled in terms of aggregated entities in the current version of the model: funding organisations and standard organisations are modelled as environmental-global variables Global variables are “global” because they are accessible by all agents and can be used anywhere in a model. Most often, global variables are used to define variables or constants that need to be used in many parts of the program.

The Supplementary Materials Section contains information on how to consult and download the model code.

3.1. From Setup to Idea Generation

After the creation and initialisation phase of the general context, the agents are characterised by specific knowledge domains. The knowledge is used to develop an innovative idea, translated into the SKIN model as the innovation hypothesis (IH). The knowledge base that characterises the agent is made visible to other agents to advertise their skills and be selected as partners. In this phase, each agent has a certain probability (this probability is defined by the value of the variable(s) characterising the agent and is named “start-project”) of being the focal agent providing the innovation idea to be developed further. The agents who can start a project have, at this point, all the resources (economic resources, knowledge base, a vague idea of the innovation) to start the idea generation phase. This phase, as we will explain better later, has been divided into three sub-phases—idea generation 1, idea generation 2, and idea generation 3—grouping the procedures that use the same theoretical framework (e.g., idea generation 3 incorporates procedures aimed at learning), preserving the sequential order of the actions of the agents and the interconnection of the sub-phases.

The agents selected by the focal agent will engage in the shared project, while those who have not been selected increase their knowledge base through research in a knowledge domain typical of their breed (incremental learning). Passage from the setup phase (the creation of the artificial world, with its characteristics and the types of agent present) to the idea generation phase is described schematically and synthetically through the flow chart in Figure 2. The mechanisms that regulate the setup of the idea generation, partner selection, and the interactions between the partners are described in the following subsections.

3.2. Idea Generation

In the idea generation phase, the interaction mechanisms aim to create and improve an innovation idea. The start-project agents are the protagonists of this phase, looking for potential partners by not only looking at past experience but also by looking for agents with adequate capabilities. A start-project agent initiates an IVC or a network since he/she does not possess all the capabilities needed for the development of the IH and some of them are also unknown; therefore, the IH in this first phase has a fuzzy and undefined character.

Agents advertise their capabilities through an advertisement so that the start-project agent can identify the best potential partners but preferring those of other breeds.

Therefore, the intersection of the advertisement of the focal agent with that of the potential partner is assessed first.

Another fundamental variable to be taken into consideration when searching for partners are their RRI inclinations, modelled as the weighted average of RRI values. The hypothesis at the basis of this computational choice depends on the evidence that a potential partner with high RRI values will be more visible to other agents and to the focal agent.

Once the minimum number of partners has been reached, the start-project agent can create an IVC/network by incurring a cost for the fine-tuning of the IH and the search for partners, while the participants will have to pay a share of contributions to constitute the common fund (investment capital). At this point, the RRI values of each member can be assessed and then averaged. This average is used to model the RRI value of the network agent and will be necessary for subsequent mechanisms of interaction with partners and exogenous agents. Members of a collective organisation must have ethical values that conform to the collective values, so a diffusion and updating mechanism of the RRI values among network participants is triggered, accounting for inertia to change.

The greater the distance between one’s own RRI values and those of the network, the lower the RRI increment and thus the willingness to conform to the collective values. The agents of the I AM RRI model can be involved in more than one innovative network at the same time; therefore, in accordance with the literature [45], they must take all the projects in which they are involved into account (especially those with ethical values similar to their own) and not only the projects with higher RRI values. This aspect avoids the creation of a “cultural bias”.

Finally, network members with sufficient economic resources can learn new knowledge from other partners through a learning and knowledge dissemination mechanism.

3.3. Gate to the Next Phase 2

The intersection between the two phases represents a filter for networks that do not possess the regulatory bodies’ ethical characteristics (ethical thinking). Networks that meet these constraints can access additional financial resources made available by the funding bodies. The requirements of these funding bodies relate to the technical quality required by the IH and the RRI inclination.

Networks that do not achieve the level of ethical thinking mandated by the regulatory bodies dissolve: network members terminate internal partnerships but keep the memory of them; the network agent disappears from the simulation environment.

3.4. Product Development

Networks that pass the first gate can begin the product development phase, which lasts a minimum of 12 ticks.

At the beginning of this phase, networks may lose members who do not have sufficient financial resources to support the promotion cost of the RRI innovation idea. A member could also parasitically exploit the resources provided by the network. Therefore, the more network resources available to support RRI values, the less effort required of the members.

Agents at this stage may decide to publish via open access, taking their financial resources and inclination towards open access into account. Once these acceptability thresholds are met, an element of randomness also intervenes, representing the various aspects of uncertainty associated with the process. This element of uncertainty is modelled through a Bernoullian variable with a parameter p equal to the open access value of the agent. Each open access publication has a cost that reduces the financial resources of the agent.

At the end of the product development phase, the mechanism of diffusion and updating of the RRI values among network members is triggered and, downstream from experiential learning, updating of expertise (taken from SKIN).

3.5. Gate to the Next Phase 3

At the end of the second phase, the networks face the second gate. At this simulation point, the networks and their innovation idea are evaluated by the regulators and standardisation organisations. The former still evaluate the ethical thinking orientation of the networks; the latter evaluate the qualitative specifications of the innovation idea.

Networks that have positive feedback during the gate to Phase 3 evaluation can create RRI-focused start-ups that operate in the market or, alternatively, dissolve.

The modalities of development and interaction just described are represented schematically and synthetically through the flow chart in Figure 3.

3.6. Start-Ups

The requirements for creating a start-up consider two aspects:

• Sufficient capital for initial support;
• Sufficiently high RRI values.

From the starting project, the start-up inherits the ethical characteristics, the innovation idea, and the knowledge base. However, the knowledge base is not cloned perfectly; in fact, the initial inexperience of the start-up must be considered: the levels of expertise are decreased by one.

At this point, start-ups are evaluated by funding bodies and can then receive a grant if their innovation idea meets the quality requirements.

3.7. Incremental Research

During the simulation, agents who have not been involved in any network try to increase their attractiveness by acquiring new knowledge through incremental searches. As a result of financial availability, a new capability is added to their kene, representing the agent’s knowledge base.

The capabilities sought in this phase are part of the set of knowledge that characterises the agent’s typology (called the “breed” in NetLogo language). The incremental research, when activated by the agent, lasts three ticks. After a new capability has been acquired, the agent makes an advertisement to try to be selected by a focal agent. If not selected again, a new research session could be initiated according to the availability of financial resources.

3.8. Outputs

The outputs are divided into three macro-areas of impact: social, economic, and strategic.

The economic performance of the system is analysed through the continuous evaluation of the average capital of the agents constituting the system and also at the end of the simulation.

The strategic performance is evaluated through several indicators:

• The number of start-ups created;
• The average size of the networks;
• The number of agents involved in the networks or the percentage of agents involved;
• The percentage of surviving agents for each phase.

The result in social terms is estimated through a graph that expresses the time trend of the average RRI value for each RRI variable identified. Thus, the increase, decrease, and periods of greatest spread of the RRI values are observed.

Another proxy used is the number of open access publications that were made during the simulation. Obviously, at each time point in the simulation, the number of networks/IVCs relative to each phase and the total number of networks is provided.

4. Validation

Validation is the process of determining how well the implemented model corresponds to reality. Validation has always played an essential role in modelling issues [46], especially concerning computational models [47,48]. We could say that the biggest problem related to validation is that there is no universally accepted approach. In the literature, there are many validation techniques and principles on which they are based, so it would be too expensive to use all the possible techniques [49].

There are different levels of validation that depend on the availability of data and on the purpose of the model and of the researchers. As suggested by Carley [47], illustrative-theoretical models usually require a reduced level of validation, while case-based models are more demanding in terms of validation due to the fact that, normally, these models are used to give practical advice on some specific aspects.

Different levels of validation identified by Carley are:

• Face validity;
• Parameter validity;
• Pattern validity;
• Process validity;
• Point validity;
• Distribution validity;
• Value validity.

The first three or four levels are more related to theoretical or illustrative models; the others are more concerned with emulative models of reality. The I AM RRI SKIN model is not a pure theoretical (illustrative) model but it is not a case-based model. Consequently, we expect to reach and intermediate level of validation.

At the moment, some data on AM in the European market are available, thanks to the presence of industrial partners in the I AM RRI project; however, these data have not been processed sufficiently to be used for a process as complex as validation. For these reasons, we can only reach a low to intermediate level of validation at this stage of the project. Following the indications of Sargent [41] and Law et al. [50], there are different ways to reach the intermediate levels of rigor approximating validation:

• Using existing theory or studies;
• Modeler experience and intuition;
• Conversations with subject-matter experts during development/face validity;
• Observing the macro-level effects.

All the approaches just listed have been used to approximate the validation process. In this paper, we focused primarily on the correspondence between existing RRI literature and the I AM RRI SKIN model’s emerging behaviours.

Below, we report on two experiments whose results may have implications for the understanding and guidance of RRI practices, at both the managerial and governance levels. In addition to the experiments reported, further experiments necessary for validation and internal verification were conducted to ensure greater rigor in the model.

5. Experimental Design and Results

Finding a correspondence in the literature on how RRI practices impact innovations is not an easy task, since “the impact appears to be elusive and difficult to measure” [51] (p. 402). In fact, “the RRI cannot be used as an evaluation tool since it does not have the material metrics to measure how responsible or positive or negative the impacts of innovation are, but is a normative framework designed to influence the process of innovation” [52] (p. 15). Thus, we can say that the RRI intervenes in the innovation process as a whole and indirectly on that process’s product.

As suggested by Gonzales-Gemio et al. [53], the performance of organisations that adopt RRI practices has benefits that can be seen in the long run, such as the acquisition of new knowledge and improving skills, processes, resources, and services aimed toward their adaptation to change [54]. Another aspect reported in the literature is that RRI practices ensure greater inclusiveness and heterogeneity in working groups [8,55,56] to reach better decisions [57], to ensure richer discussion [58], and to ensure greater impact of the sustainable innovations produced by collaborative networks [53,59].

We can conclude that the introduction of RRI practices influences the innovation process and the actors involved in two ways:

• Increased heterogeneity of the actors involved in the innovation project;
• An increased knowledge base.

Based on the literature on the impact of adopting RRI practices in innovation processes, we established an experiment to validate the correspondence between the arguments reported in the literature and the I AM RRI SKIN model’s behaviour.

Thus, we wondered if agents gave more importance to RRI practices and selected their partners considering RRI principles, we would observe an increase in the heterogeneity of the actors involved in networks. Will the knowledge base also increase at the end of the simulation? The variable of interest, as we can guess, is the threshold of the attractiveness of partners based on RRI values (RRI attractiveness), while the outputs inserted are as follows:

-

Average Heterogeneity of Networks, where network heterogeneity was measured as the ratio of the number of different breeds within the network to the total number of members.

$$Heterogeneity=\frac{{{\displaystyle \sum }}_{breed\in Network}breed}{number of partners}$$

(1)

-

Average Knowledge of Agent. Starting from the “kene”, the knowledge $\left(k_i\right)$ of the agent was modelled, taking the length of the capabilities vector into account. We could not use the expression for the knowledge of the i-th agent used by Ahrweiler et al. [60] (p. 227):

$$k_i=\sqrt{{\displaystyle \sum }_{j\in {\left\{C\right\}}_i}{A}_{ij}{E}_{ij}}$$

(2)

In the I AM RRI SKIN model, the abilities of the agents do not belong to an ordinal scale but represent the label of the capability in the broader knowledge domain. To be clearer, in I AM RRI SKIN, Ability 6 is not “better” than Ability 3. The knowledge of the i-th agent is calculated by the following expression, starting from the agent’s kene.

Finally, the average knowledge of the agents participating in an innovative project can be represented as:

$$\overline{k}=\frac{1}{nFirms}{\displaystyle \sum }_{i\in \left\{partnering Firms\right\}}{k}_i$$

(3)

As mentioned, we used RRI attractiveness as the input variable; all other independent variables were considered as control variables and are reported in

Table 1. Experimental design.

Control Variables
Numbers of Agents		Firm Variables		Environmental Variables
n AM-tech	200	Attractiveness threshold	0.5	Standard organisation	5
n Supplier	200	RRI start-up trigger	0.5	Funding	50
n Research-inst.	200	Publish open access		Funding RRI	0.5
				Funding quality	5
n OEM	200	Economic threshold	50	RRI cost	30
n Customer	200	RRI open access threshold	0.5	Big firm percent	10
				Regulator	0.5
Setting, Explanatory Variable, and Output
Run	Ticks	RRI attractiveness		Output
300	30	0.2, 0.5, 0.8		Average heterogeneity of networks Average knowledge of the agents

, along with the combination of parameters used to set up the experiments.

The input variable RRI attractiveness, which expresses the weight given to RRI values in the partner selection process, has been divided into three ranges: low [0–0.3], medium [0.4–0.6], and high [0.7–1]. The value 1 indicates that the partner selection process is based exclusively on the compatibility of RRI values, while a value of 0 indicates that the selection of partners is based exclusively on the complementarity of the knowledge base.

To investigate the effect of the importance given to the RRI values during the selection of the partners on the heterogeneity of the networks, we first made use of some descriptive statistics concerning the average heterogeneity of the networks when varying the RRI attractiveness factor. The results reported in

Table 2. Descriptive statistics: Experiment 1.

Descriptive Statistics
RRI Attractiveness	N	Mean	Std. Dev.	Std. Error	95% Confidence Interval for Mean		Min	Max
					Lower Bound	Upper Bound
0.2	300	0.6553	0.02982	0.00172	0.6519	0.6587	0.57	0.74
0.5	300	0.6758	0.03027	0.00175	0.6724	0.6793	0.57	0.76
0.8	300	0.6909	0.03017	0.00174	0.6875	0.6944	0.60	0.77
Total	900	0.6740	0.03342	0.00111	0.6718	0.6762	0.57	0.77

refer to the 30-th tick of the simulation. For each level of RRI attractiveness (0.2, 0.5, and 0.8), 300 runs were performed.

The current literature shows that an increase in the diffusion and importance given to RRI practices corresponds to an increase in heterogeneity within innovation systems. The scatterplot in Figure 4 shows that the I AM RRI SKIN model behaves as expected from the literature [8,55,56].

However, to consider significant the effect of the RRI attractiveness factor on the dependent variable’s heterogeneity, a one-way ANOVA was used, the results of which are F(2, 897) = 106.03, p < 0.0001.

We can reject the null hypothesis $H_0$ by accepting a risk of the first kind at $\alpha =0.05$. Thus, we can consider the influence of the RRI attractiveness factor on network heterogeneity to be significant in our model.

In order to further deepen the analysis, post-hoc tests (Tukey’s HSD) were performed following the significance of the ANOVA, whose results are reported in

Table 3. Post-hoc: Experiment 1.

	(I) RRI Attractiveness	(J) RRI Attractiveness	Mean Difference (I-J)	Std. Error	Sig.	95% Confidence Interval
						Lower Bound	Upper Bound
Tukey’s HSD	0.2	0.5	−0.02053 *	0.0024	0.000	−0.0263	−0.0148
		0.8	−0.03563 *	0.0024	0.000	−0.0414	−0.0299
	0.5	0.2	0.02053 *	0.0024	0.000	0.0148	0.0263
		0.8	−0.01510 *	0.0024	0.000	−0.0209	−0.0093
	0.8	0.2	0.03563 *	0.0024	0.000	0.0299	0.0414
		0.5	0.01510 *	0.0024	0.000	0.0093	0.0209

* The mean difference is significant at the 0.05 level.

.

These tests, performed two by two, offer us experimental evidence that the average difference is statistically significant. Therefore, observing that all three levels of RRI attractiveness produce statistically significant effects, we can consider that this initial subdivision into three ranges (low, medium, high) is acceptable.

The second experiment aimed to reinvestigate the effect of a policy of selecting partners by focusing on the RRI values, so the input variable remains the RRI attractiveness. We try to establish whether an increase in this variable led to an increase in the system’s knowledge base. In this case, the variable of output was the average knowledge of the agents, as described previously.

As mentioned, the existing literature on RRI indicates that an increase in RRI practices should be matched by an increase in the knowledge base of the agents. To test this hypothesis, a one-way ANOVA was used, setting the parameters as described above (Table 1). The ANOVA results—F(2, 897) = 1.448, p = 0.236—suggest that there is no experimental evidence to suggest that the effect of RRI attractiveness on the agents’ knowledge is significant. Post-hoc tests (Tukey’s HSD) were performed following the significance of the ANOVA, whose results are reported in

Table 4. Post-hoc: Experiment 2.

	(I) RRI Attractiveness	(J) RRI Attractiveness	Mean Difference (I-J)	Std. Error	Sig.	95% Confidence Interval
						Lower Bound	Upper Bound
Tukey’s HSD	0.2	0.5	0.00156	0.00361	0.999	−0.0832	0.0086
		0.8	−0.0052	0.00361	0.315	−0.0137	0.0032
	0.5	0.2	−0.0001	0.00361	0.999	−0.0086	0.0083
		0.8	−0.0053	0.00361	0.294	−0.0138	0.0030
	0.8	0.2	0.0052	0.00361	0.315	−0.0032	0.0137
		0.5	0.0053	0.00361	0.294	−0.0030	0.0138

.

At this point, the investigation did not stop. First, the code was modified so that the advertisements were formed only by the IH’s capabilities, as in SKIN [34]. This modified the mechanism of partner selection and learning. However, even in this case, the ANOVA was not significant—F(2, 897) = 1.206, p = 0.300.

6. Discussion

Finding a correspondence in the literature on how RRI practices impact innovations is not an easy task, since “the impact appears to be elusive and difficult to measure” [51] (p. 402). In fact, “the RRI cannot be used as an evaluation tool since it does not have the material metrics to measure how responsible or positive or negative the impacts of innovation are, but is a normative framework designed to influence the process of innovation” [52] (p. 15). Thus, we can say that RRI, being a normative framework represented by new guidelines and a new paradigm to follow, intervenes on the innovation process as a whole and indirectly on that process’s product.

Although there are still disagreements between academia and administration on the definition of RRI [7], it seems clear that among the purposes of RRI practices, societal involvement plays an essential role [7,61,62]. Kupper et al. [56] translated this concept by formulating the process requirements of diversity and inclusion, measured in our model through the variable heterogeneity. The heterogeneity of stakeholders, i.e., the presence of different points of view, is a critical feature in innovative adaptive processes [63] to reach better decisions [57] and to ensure richer discussions [58]. Ultimately, an innovative project guided by RRI principles must be characterised by greater heterogeneity in the working group. Conclusion, which is closely linked to the concept of inclusion, is a cornerstone of RRI practices [64].

The internal validation carried out thought the proposed experiment demonstrated emergent behaviour in complete alignment with the principles just described while also demonstrating that the conceptual model on which it was based is correct. As Kupper et al. [56] suggested, heterogeneity must permeate all stages of the RRI process, starting with the initial stages. In our model, this aspect was emphasised by the importance (greater or lesser, depending on the parameters chosen for the simulation) given to the potential partners’ RRI values during the search and selection phase. As shown by the ANOVA results of post-hoc multiple comparisons, greater importance given to RRI values during partner selection corresponded to an increase in network heterogeneity. In this way, diversity—the inclusion of different types of stakeholder—becomes intrinsic to the importance given to the RRI values modelled as exogenous agent variables. Ultimately, a selection process that also considers the RRI values of the agents, and not only the type of knowledge base and financial resources of potential partners, implies a significant increase in network heterogeneity.

Some authors measured the performance of organisations that adopt RRI practices from a financial point of view (measuring, for example, the level of sales growth, the level of return on equity, ROA, market share, or the level of productivity), while others focus on a non-financial point of view, such as an increase in the knowledge base. Finally, the long-term benefits are observed mainly of an internal nature, such as “the development of new resources and capabilities” [53] (p. 17) and an increase in knowledge that leads to more excellent responsiveness in the identification of potential innovations [55].

In our model, agents have a particular knowledge base depending on the type of breed they belong to, resulting from the analysis of various use-cases provided by the partners of the I AM RRI project. This knowledge is spread and exchanged among the various members of the network during the simulation.

The results of the one-way ANOVA led us to believe that the greater importance of RRI values during partner selection was not significant. The mismatch with the literature can be explained by the small number of capabilities present in the system (nine in total). For this reason, an agent probably already possesses the capability of his/her partner, making learning unnecessary. Future developments can first address the identification of a larger number of capabilities present within the AM industry through collaboration with the I AM RRI project’s industrial partners.

These initial results represent a new contribution to the literature on sustainable innovation processes and are an alternative way to communicate the benefits of adopting RRI practices. The proposed computational model represents a simulative tool for identifying the best management strategies. While the literature provides useful insights into the consequences of sustainable innovation processes, our model, starting from a conceptual model developed within the I AM RRI project, confirms them. Besides, the discussion can be enriched with new research questions about RRI practices that can be explored through our model.

7. Conclusions

The use of an ABM enabled the modelling of innovation systems without giving up essential elements of reality (e.g., heterogeneous agents, uncertainty in interactions with the environment and other agents, experimental learning, and networking).

I AM RRI SKIN constitutes a first attempt to create an auxiliary tool for institutional bodies and policymakers, assisting them in defining strategic guidelines for disseminating and encouraging RRI best practices. Although the conceptual model is inspired by some use-cases provided by partners of the I AM RRI project operating in the AM automotive or biomedical industry, the model is easily adaptable to other contexts or industries, where cooperation between innovation network members also has a responsible component. Therefore, after identifying the actors, mechanisms, and knowledge assets, it is possible to proceed to the extension/integration of the proposed model.

The internal validation of our model has been tested through different simulation scenarios, the results of which demonstrated a correspondence between the artificial I AM RRI SKIN world and the existing RRI literature. The model offers an understanding of IVC networks and the potential of RRI particles, and allows us to simulate the processes of dissemination and knowledge creation. For this reason, scrupulous research about the types of knowledge present in the AM industry was undertaken with I AM RRI project partners, to increase the size of the agents’ kene. In addition, new types of agent that intervene in the innovation process, such as NGOs, and new RRI variables such as gender equality and science education will be included in the model. Thus, the next steps will be oriented towards a full validation, starting from the processing of data made available by the project partners, and using empirical data to set the input parameters and compare the simulation results with the empirical evidence.