Measuring the impact of technological innovations on industrial products through a multilayer network approach

Measuring the impact of technological innovations on industrial products through a multilayer network approach Martina Formichini1, Giulio Cimini2,3, Emanuele Pugliese4, Andrea Gabrielli3 1 Dipartimento di Fisica, “Sapienza" Università di Roma, Piazzale A. Moro 5, 00185 Rome, Italy 2 IMT School for Advanced Studies, Piazza S. Ponziano, 6 55100 Lucca (Italy) 3 Istituto dei Sistemi Complessi (ISC) CNR, UoS Sapienza, Dipartimento di Fisica, “Sapienza" Università di Roma, Piazzale A. Moro 5, 00185 Rome, Italy 4 EC JRC, Calle Inca Garcilaso 3, E-41092 Seville (Spain) Version August 4, 2018 submitted to


Introduction
Technological innovation is the main driver of modern economic growth [1,2].It is therefore not surprising that measuring and predicting the potential impact of technological innovations on export competitiveness has been the central issue of many studies in the last forty years [3,4] and the focus of a general interest in the field of innovation systems.Such academic effort provided a theoretical framework and empirical stylized facts helpful to understand the crucial effect of innovation in shaping competitive advantage of countries in different markets.Policy makers are however more interested in identifying the specific technologies that are relevant for specific markets [5,6].These questions are much harder to answer in an organic and objective fashion: the scientific approach to address the impact of specific technologies on specific markets has been through ad-hoc case studies of difficult comparability.
In a recent paper [7] a multilayer network approach to the innovation system has been proposed: starting from the three bipartite networks countries-products, countries-technologies and countries-scientific sectors, with arbitrary relative time delay, a three-layered network of innovation activities is derived in which it is possible to measure the conditional probability that a bit of information produced in an innovation activity (e.g. a scientific domain as for instance microelectronics) will impact after a given delay on another innovation activity (e.g. an industrial product category).By grounding on previous fundamental studies of Economic Fitness and Complexity [8,9], this is the first approach representing the innovation system as a complex system and using bipartite and multilayer network, diffusion theory on networks and maximum entropy network ensembles for its study.
In this paper we adopt a similar approach, by limiting our analysis to the relationships between technologies and products, but generalizing the method so that we can measure the potential impact of a group of technological sectors (in particular pairs of sectors) on single product categories.This is indeed a crucial aspect of technology: the interaction between different technological fields is often a crucial aspect of technological progress [10] We use the following established databases to construct the bi-layered technologies-products space: for Technology, we consider the number of patents in different technological sectors extracted from Patstat (www.epo.org/searching− for − patents/business/patstat); and for Products, we Results are then statistically validated by building appropriate advanced null models derived from the constrained maximum entropy Bipartite Configuration Models (BiCM) for both the bipartite countries-technologies and the countries-products networks.

General approach and strategy
Starting from Patstat and UN COMTRADE databases, we can define the Revealed Comparative Advantage of a country c on an activity a (which has to be meant either a technological sector t or and product category p) in a given year y as where in the case of a technology t, the quantity W c,t (y) is the number of patents of country c in the technological sector t in the year y, while in the case of products W c,p (y) we use, as a proxy of the activity of a country c in a product category p in the year y, the amount of dollars coming to the same country by the export of that product in that year.The years interval considered in this work 1995 − 2012, the index c spans over all the considered countries whose total number N c varies from 66 to 72 depending on the year, t over the total set of technological sectors whose number N t veries between a minimum number 629 and a maximum 636 also depending on the year, and p spans over the total set of considered product categories N p which varies between 1140 and 1176 (note that slight variations of N c , N t and N p year by year are due to geographical changes and periodical recategorization of technologies and products).
It is possible to define for a given year y the binary bipartite networks countries-technologies and countries-products respectively represented by the binary biadjacency matrices M C,T (y) and M C,P (y) whose elements M ct (y) and M cp (y) are defined as follows: where again a refers to a technological sector t in the case of the bipartite countries-technologies network and to a product category p in the countries-product case.
Once we have the matrices M C,T (y 1 ) for the year y 1 and M C,P (y 2 ) for the year y 2 , in analogy with [7], we can construct the assist matrix B T →P (y 1 , y 2 ), whose generic element is defined as: where k t (y 1 ) = ∑ c M ct (y 1 ) is the number of countries having the technology t in their technological basket of year y 1 , k is the cardinality of the product basket of country c in the year y 2 .As well explained in [7], B tp (y 1 , y 2 ) with y 1 ≤ y 2 gives the conditional probability that a bit of information produced in the technological sector t in the year y 1 arrives at the product category p in the year y 2 (through one of the countries having t in its technological basket of the year y 1 and p in the product basket of the year y 2 ).Note that the elements B tp (y 1 , y 2 ) can be also seen as the weights of the links of the bipartite network technologies-products with the former at year y 1 and the latter at year y 2 whose probabilistic interpretation has just been given above.
Through the assist matrix B T →P (y 1 , y 2 ), we define the Λ motifs of such bipartite network as: can also seen as the joint and conditional probability that two bits of information produced in the technological sectors t and t in the year y 1 arrives at the product category p in the year y 2 (through one of the countries having t and t in its technological basket of the year y 1 and p in the product basket of the year y 2 ).The name Λ−motif comes from the fact that in the aforementioned bipartite network technologies-products between the years y 1 and y 2 , Eq. ( 4) gives the weight of the a Λ shaped set of two links having different origin in the technology layer and same end in the product layer [11].
Thus in principle starting from the biadjacency matrices M C,T (y 1 ) and M C,P (y 2 ) extracted directly In principle this quantity can be even easily generalized to assess the impact of a wider group of technologies on a single product, but for aim of simplicity and statiostical significance we will limit here our analysis to pairs of technologies and single products.
Once we evaluate the probability Λ tt (∆y) from real datasets, we statistically validate it by comparing its value with the probability distribution of the values taken by the analogous quantity in an appropriate null model.
Such a null model is schematically defined as follows (see Appendix for a thorough presentation): for each year y we build two statistical ensembles of biadjacency matrices MC,T (y) and MC,P (y) entropy approach in the spirit of the information theory approach to the statistical mechanical ensembles [12].
These ensemble are called in literature Bipartite Configuration Models (BiCM) [13].Finally, given the appropriate BiCMs for the all bipartite networks MC,T (y 1 ) and MC,P (y 2 ) with fixed y 2 − y 1 = ∆y, we can derive, through Eqs. ( 3) and ( 4) appropriately applied to the BiCMs, the probability distribution of the value Λp tt (∆y) of the joint probability that two bits of information, started a given year from the technologies t and t , arrive at the product p after a time delay ∆y.
As explained in the section below, by comparing in terms of fixed statistical significance value α = 0.01 the values of the probabilities Λ p tt (∆y) obtained by the real bipartite networks with respect to the distribution of the values of the same quantity in the null model, we can finally validate the real impact of the pair of technologies on single products after an arbitrary delay.

Results
As mentioned in the previous section, the empirical assist matrices B T →P (y 1 , y 2 ) we consider are made up of a number between 629 and 636 of technological sectors (at a 4-digits resolution) and a number between 1140 and 1176 products (also at a 4-digits resolution) depending on the years y 1 and y 2 .
Years y 1 and y 2 which refer to matrices of countries-technologies M C,T (y 1 ) and of countries-products M C,P (y 2 ) respectively both range from 1995 to 2012.The null case for these matrices are built by numerically generating two ensemble of 10 3 matrices MC,T (y 1 ) and MC,P (y 2 ) using the BiCM, and then contracting each pair to generate a final ensemble of 10 3 null matrices BT →P (y 1 , y 2 ).As explicitely described in the previous section, we define Λ motifs of the assist matrix by Eq. ( 4) and then we average the values of Λ ∆y between technologies and product in the capabilities of countries.We define the signal φ(∆y) as the fraction of significant Λ p tt (∆y) (at the α = 0.01 significance level) for combinations of t, t and p chosen for selected matrix regions.The exact procedure followed to generate a null model to determine the statistical significance of the observed Λ p tt (∆y) is explained in the Appendix 5.In general we consider a population of motifs equal to 5500 units (see below).
As a first test we report the mean signal φ, i.e., the signal averaged over all combinations of t, t and p.Since the total number of such motifs is extremely large, we choose 5500 motifs at random and take their signal as representative of the mean one.Figure 1 shows that φ basically remains within one standard deviation from the noise level α, indicating that the mean signal within the data is negligible.We then report the signal relative to motifs within selected regions of the assist matrix.Specifically, we choose sub-regions of 11 technologies and 100 products (related to specific technology and production fields), whose total number of (unordered) Λ motifs is 5500.From Figure 2 we see that, by selecting coherent sets of technologies and products, the signal is much enhanced: the presence of a pair of technologies in the capability basket of a country can predict if that country can successfully export a product, and this happens almost independently on the time lag ∆y.
As consistency checks we make two exercises, both reported in Figure 3. Firstly, we show that the results we just presented do not depend on the particular resolution used to choose the motifs.
Secondly, we show that for incoherent technologies and products, we can indeed get a much lower signal-even lower than the significance level.In this latter case, a significant development of specific technologies corresponds to a low-level export of given products.
We finally want to provide a few examples of motifs with high signal.To do that, since the total number of motifs is extremely high so that a complete exploration is not efficient and surely not clever, we checked the motifs made up of the link pairs B tp (y 1 , y 2 ) and B t p (y 1 , y 2 ) which are independently the most significant.Table 1 reports some instances of such motifs for the specific choice ∆y = 0.

Conclusions
In this work we provided an effectual way of measuring the combined effect of a set of technology on one product.In particular in this work we highlight lambda motifs: the paired effect of two technology together.
In the process of finding relevant combinations, we highlight several results.we confirm that regional co-occurrences between activities are able to extract information on shared capabilities, to inform policy makers and stakeholders of relevant synergies.
The mapping provided by the techniques of the effects of pairs of technologies on products can be transformed in a powerful instrument to inform policies and industrial strategies.This operational step will be an important aspect of future research.

Appendix: the Bipartite Configuration Model (BiCM) and the null model
In order to assess the statistical significance of elements of the assist matrices, we resort to a null model for the bipartite matrices {M C,T (y), M C,P (y)}, built by randomly reshuffling their elements (i.e., the network links connecting nodes in the layer C of countries with respectively nodes the layer T of technologies and P of products), but preserving country diversifications and activity (i.e., technologies or products) ubiquities in the two bipartite networks (i.e., the degrees of the nodes in both layers of the bipartite networks countries-technologies and countries-products) .This means that we randomize the signal coming from the high order network connectivity patters, beyond that encoded by the node degree.In order to formulate the null model analytically, avoid relying on a conditional uniform graph test [14,15], degree constraints are imposed on average, in a formally similar way to what happens in the Canonical ensemble in Statistical Mechanics with the constraint of the energy [12].This coincide to Left panel: technological codes related to sector "chemistry" -C07 e C30 (i.e., a much larger region than that represented in the bottom panel of Figure 2) and products in the region 2706-3104 inorganic and organic chemicals, pharmaceuticals.Right panel: technological codes related to sector "physics" -in the region H01P-H02M electricity, and products in the region 4411-5516 textiles.In all cases, error bars represent the standard deviation over the year pairs giving the same time lag, whereas, the dotted line is the significance level α.
set a null hypothesis described by the Bipartite Configuration Model (BiCM) [13], which is an extension of the Configuration Model [16], originally introduced by Park and Newman, to bipartite networks.
From a general point of view, the BiCM null model for any given (i.e., real) binary biadjacency matrix M, representing a bipartite network with two layers of node a ∈ A and b ∈ B, is defined as the ensemble Ω of bipartite networks which are maximally random, apart from the ensemble average of the degrees of the nodes of both layers of the bipartite network which are constrained to appropriate values.As shown below, by a maximal likelihood argument, it is possible to show these appropriate values coincide with observed ones in the empirical network [17]: ka Ω = k a ∀a ∈ A and kb Ω = k b ∀b ∈ B, where we have indicated with k the observed degrees in the real network and with k the degrees in a generic configuration of the null model.We remind that k a = ∑ b M ab and k b = ∑ a M ab and analogously for "tilded" quantities.In the following, we use symbols with the tilde for quantities assessed on null model configurations, and without the tilde for observed values.
As we explicitly described below, such BiCM is operationally defined by two-steps: (i) by a constrained maximum entropy approach, we introduce the Exponential Random Bipartite Graph (ERBG) ensemble { M} with arbitrarily fixed mean values of the degrees of the nodes of the two layers; (ii) in order to choose among all possible ERBGs the best null model for our real bipartite network M, we set the mean values of the node degrees equal to the values that they take in the real network M by a maximum likelihood argument.
Let us start by introducing the ERBG with fixed mean node degrees and let M ∈ Ω be a network configuration in such ensemble and P( M) be the probability of that graph.By implementing the prescriptions from Information Theory and Statistical Mechanics [12,18], the least biased choice of P( M) is the one that maximizes the informational entropy subject to the normalization condition ∑ M∈Ω P( M) = 1 plus the constraints: where k * a ∀a ∈ A and k * b ∀b ∈ B are arbitrarily fixed values for the mean degrees of nodes belonging to layers A and B. By defining the respective Lagrange multipliers ω, {µ a } a∈A and {ν b } b∈B (one for each node of the bipartite network), the probability distribution that maximizes the entropy satisfies satisfying at the same time the constraints, for all configurations M ∈ Ω is given by the following variational equation: It is a matter of simple algebra to show that the solution of this equation is: where the function H( M | {µ a }, {ν b }) is usually called the Hamiltonian of the graph configurations and Z({µ a }, {ν b }) is the corresponding partition function The above Eqs.( 8), ( 9) and ( 10) define the network ensemble known as the BiCM model.
Note that, as we have implemented only local constraints, i.e. the node degrees, Eq. ( 8) can be rewritten as the product of single link probability distributions over all pair of nodes belonging respectively to the two different layers [13]: where π ab is simply the probability of the link between nodes a ∈ A and b ∈ B: with η a = e −µ a and θ b = e −ν b .In other words the existence of different links are independent events with respective probabilities related only to the Lagrange multipliers of the lateral nodes of the links.The values of the Lagrange multipliers are determined by the constraints Eqs. ( 6) which can be rewritten in terms of the derivatives of the partition function: ) Equations ( 8)-( 12) define the generic EBRG with fixe mean degrees of nodes on both layers of the bipartite network.
At this point, in order to obtain the optimal null model for a given real biadjacency matrix (i.e. a bipartite network) M among all possible above defined EBRGs, we have to choose the best values for To this aim we write the log-likelihood function [17] L({µ a }, {ν b }) where P(M | {µ c }, {ν a }) is the probability measure (11) Finally, the null model for a real bipartite network M is defined by Eqs. ( 11) and ( 12) with the Lagrange multipliers set by Eqs.(15).This recipe can therefore be applied to construct an appropriate null model for all empirical bipartite networks {M C,T (y), M C,P (y)} y min ≤y≤y max obtained respectively by the databases Patstat and COMTRADE.
In order to build a null model for the assist matrices B T →P (y 1 , y 2 ), and consequently for the Λ motifs defined by Eq. ( 4), obtained by Patstat and COMTRADE data at different years y 1 and y 2 , we can now compose null models for the bipartite networks M C,T (y 1 ) and M C,P (y 2 ).This is done by contracting the two BiCMs for the matrices M C,T (y 1 ) and M C,P (y 2 ) along the country dimension [11,19], as for Eq. ( 3).We have: where kt (y 1 ) = ∑ c Mct (y 1 ) and k(p) c (y 2 ) = ∑ p Mcp (y 2 ) are respectively the ubiquity of technology t and the product diversification of country c in the two single configurations for the BiCM null models for the two bipartite networks countries-technologies of year y 1 and countries-products of year y 2 .In other words, starting by the two BiCM ensembles for M C,T (y 1 ) and M C,P (y 2 ) we build by composition an ensemble of configurations of bipartite networks Ω T →P (y 1 , y 2 ) with link weights given by Eq. ( 16).The distributions of such Btp (y 1 , y 2 ) values describing the null model can be in principle obtained using exact techniques [11,19].However due to the non-Gaussianity of such distributions, we adopt a more practical sampling technique: starting from the BiCMs for M C,T (y 1 ) and M C,P (y 2 ), we use Eqs.(11), (12) and ( 16) to generate null Assist matrix bipartite networks, and populate the related ensemble Ω T →P (y 1 , y 2 ) to estimate the full distributions.In a similar way, by using the composition Eq. ( 4) for the ensemble Ω T →P (y 1 , y 2 ): and averaging over all pairs of years y 1 and y 2 with fixed delay ∆y = y 2 − y 1 to get Λtp (∆y), we can construct the null distribution of Λ motifs for each triple t, t , p and delay ∆y.
The generic observed element Λ p tt (∆y) is then considered statistically significant depending on the p-value that we can infer from its distribution under the null hypothesis.The specific threshold for statistical significance and the size of the generated ensemble vary on the exercises performed (as highlighted in the text).In our case we fixed the statistical significance level at α = 0.01.It is useful to recall that the two choices, the threshold and the size of the ensemble, are not unrelated: the higher the threshold we want to test, the bigger the sample we require.We consequently extracted for each couple of years y 1 and y 2 two ensemble of 1000 configurations/matrices for the two BiCM null models M C,T (y 1 ) and M C,P (y 2 ), and contracting one to one the configurations in the two ensembles, we get from the databases Patstat and UN COMTRADE at different respective years y 1 and y 2 , one can measure the complex probability Λ p tt (∆y) of impact of pairs of technologies on a product category after an arbitrary time-delay ∆y, by averaging up all Λ p tt (y 1 , y 2 ) with fixes years difference ∆y = y 2 − y 1 .
for the bipartite networks countries-technologies and countries-products which are maximally random apart from fixing the ensemble mean values of the degrees of the nodes of such bipartite networks (i.e. the technological diversification of countries k(t) c (y) = ∑ t My ct and the technologies ubiquities kt (y) = ∑ c My ct for the first bipartite network and the product diversification of countries k(p) c (y) = ∑ p My cp and the product ubiquities kp (y) = ∑ c My cp for the second bipartite network) equal to the values in the real matrices.The probability measure of both statistical ensembles of binary bipartite networks is, as explicitly shown in Appendix through a constrained maximum p tt (y 1 , y 2 ) over all pairs of years with the same difference y 2 − y 1 = ∆y in order to get Λ p tt (∆y) representing the co-occurrences of two technologies t, t and one product p with a time delay Preprints (www.preprints.org)| NOT PEER-REVIEWED | Posted: 4 August 2018 doi:10.20944/preprints201808.0081.v1

Figure 1 .
Figure 1.Mean signal φ of 5500 combinations of t, t and p chosen at random, for different values of the time lag ∆y.Error bars represent the standard deviation over the year pairs giving the same time lag, whereas, the dotted line is the significance level α.

Figure 2 .
Figure 2. Mean signal φ of 5500 combinations of t, t and p chosen for specific region of the assist matrix.Top left panel: technological codes related to sector "physics" -in the region G09F-G11B instruments of communications, acoustics, optics, and products in the region 8401-8518 machinery and metals.Top right panel: technological codes related to sector "engineering" -in the region F22G-F23N various types of machines including steam and combustion, and products in the region 8401-8518 machinery and metals.Bottom panel: technological codes related to sector "chemistry" -C09H e C10L macromolecular and inorganic compounds, gas and petroleum, products, and products in the region 2706-3104 inorganic and organic chemicals, pharmaceuticals.In all cases, error bars represent the standard deviation over the year pairs giving the same time lag, whereas, the dotted line is the significance level α.

PreprintsFigure 3 .
Figure 3. Mean signal φ of 5500 combinations of t, t and p chosen for specific region of the assist matrix.Left panel: technological codes related to sector "chemistry" -C07 e C30 (i.e., a much larger region than that represented in the bottom panel of Figure2) and products in the region 2706-3104 inorganic and organic chemicals, pharmaceuticals.Right panel: technological codes related to sector "physics" -in the region H01P-H02M electricity, and products in the region 4411-5516 textiles.In all cases, error bars represent the standard deviation over the year pairs giving the same time lag, whereas, the dotted line is the significance level α.

Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 4 August 2018 doi:10.20944/preprints201808.0081.v1 which
[7]sures the joint and conditional probability of co-occurrence in a single country of a pair of technologies t and t at year y 1 and of a product p at year y 2 .As explicitly shown in[7]Λ

4 August 2018 doi:10.20944/preprints201808.0081.v1Table 1 .
Sample of top-significant Λ motifs.The p-value is averaged over all year pairs y 1 = y 2 giving ∆y = 0. To make this selection, we picked the most significant pairs (individual links) (t, p) and then choose t within the region where the average p-value was highest.
First of all, we show how the combination of multiple technologies has a very different role in different industrial and technological sectors.This heterogeneity, while expected, is here quantitatively measured.Secondly, Preprints (www.preprints.org)| NOT PEER-REVIEWED | Posted: evaluated for a configuration coinciding with the real network M and {k a } a∈A and {k b } b∈B are the node degrees of M. The best values for {η a } a∈A and {θ b } b∈B (or equivalently {µ a } a∈A and {ν b } b∈B ) are therefore obtained by maximizing such log-likelihood in these parameters.It is simple to show that this amounts in nsolving the system of |A| + |B| equations in |A| + |B| unknowns: