SMGen: A Generator of Synthetic Models of Biochemical Reaction Networks

Abstract

Several software tools for the simulation and analysis of biochemical reaction networks have been developed in the last decades; however, assessing and comparing their computational performance in executing the typical tasks of computational systems biology can be limited by the lack of a standardized benchmarking approach. To overcome these limitations, we propose here a novel tool, named SMGen, designed to automatically generate synthetic models of reaction networks that, by construction, are characterized by relevant features (e.g., system connectivity and reaction discreteness) and non-trivial emergent dynamics of real biochemical networks. The generation of synthetic models in SMGen is based on the definition of an undirected graph consisting of a single connected component that, generally, results in a computationally demanding task; to speed up the overall process, SMGen exploits a main–worker paradigm. SMGen is also provided with a user-friendly graphical user interface, which allows the user to easily set up all the parameters required to generate a set of synthetic models with any number of reactions and species. We analysed the computational performance of SMGen by generating batches of symmetric and asymmetric reaction-based models (RBMs) of increasing size, showing how a different number of reactions and/or species affects the generation time. Our results show that when the number of reactions is higher than the number of species, SMGen has to identify and correct a large number of errors during the creation process of the RBMs, a circumstance that increases the running time. Still, SMGen can generate synthetic models with hundreds of species and reactions in less than 7 s.

1. Introduction

Systems biology is a multidisciplinary research field that combines mathematical, computational, and experimental expertise to understand and predict the behaviour of complex biological systems [1,2]. Among the different formalisms that can be used to describe intracellular processes, reaction-based models (RBMs) [3,4,5] are the most suitable for obtaining a detailed comprehension of the mechanisms that control the emergent behaviour of the system under analysis [4]. The analysis of RBMs can be used to drive the design of focused lab experiments; to this aim, computational tasks such as parameter estimation (PE), sensitivity analysis (SA), and parameter sweep analysis (PSA) are generally applied [1,5,6,7]. Unfortunately, these computational tasks require the execution of huge amounts of simulations, so that the capabilities of biochemical simulators running on central processing units (CPUs) (see, e.g., [8,9,10]) can be easily overtaken. Thus, several simulators exploiting graphics processing units (GPUs) have been lately introduced to reduce the running times (see, e.g., [11,12,13,14,15,16,17,18,19]).

A crucial point, whenever new simulators are designed and implemented, regards the evaluation of their computational performance and their efficiency in executing the aforementioned demanding tasks. In this context, RBMs represent a key means as they can be exploited to run both stochastic simulation algorithms and (deterministic) numerical integration methods. Although there are more than 1000 models of real biochemical systems publicly available on BioModels [20,21], some of which characterized by hundreds of species and reactions, the majority of these models do not follow the law of mass-action. In addition, these models do not have the structural characteristics (e.g., the number of species and the number of reactions) necessary to perform a fair comparison among the simulators under different scenarios. Only a limited number of RBMs is present in the literature (e.g., signal transduction pathways [22,23,24] or metabolic pathways [25]). Thus, the lack of detailed RBMs, especially those characterized by hundreds or thousands of reactions and molecular species, hampers the possibility of performing a thorough analysis of the performance of these simulators.

The computational performance of several GPU-powered tools has already been assessed using randomly generated synthetic RBMs [13,18,19]. However, only a few generators of biochemical models have been proposed so far, hindering the possibility of having a common and well-defined benchmarking approach. For instance, Komarov et al. [13,14] developed a tool to generate synthetic networks, which was then used to test the performance of their GPU-based simulators. Given the number of reactants, the type of reactions to be included in the RBM, and the total number of reactions, they generated synthetic RBMs by exploiting a hash table to avoid duplicates. The tool was then modified by randomly sampling the values of the initial concentrations of the species from a uniform distribution and the kinetic constants from a logarithmic distribution [18]. Another known and established model generator is the reaction mechanism generator (RMG) [26], which was specifically developed to create synthetic chemical processes. RMG exploits an extensible set of 45 reaction families to generate elementary reactions from chemical species, while the reaction rates are estimated using a database of known rate rules and reaction templates. RMG relies on graphs to represent the chemical structures and trees to represent thermodynamic and kinetic data. Due to its peculiarities, RMG was used to, e.g., automatically create kinetic models for the conversion of bio-oil to syngas through gasification [27]. Finally, other tools, such as Moleculizer [28], were introduced for the generation of reaction systems to obtain a deeper understanding of transduction networks.

Despite the efforts done to automatically define synthetic models, all these generators share a common drawback, that is, they have a limited flexibility and can generate only a restricted set of biochemical networks and processes. As a matter of fact, the existing methods generally do not perform any check on the generated models, while some of them can only be used to create a restricted set of models (e.g., RMG can generate only elementary reactions). Moreover, in the case of the approach proposed by Komarov et al., even though it can generate first- and second-order reactions, including degradation and production reactions—which are the most common reactions in real biochemical networks—the tool is not publicly available.

Considering the impelling necessity of defining a common benchmarking approach that allows for fairly evaluating and comparing different simulation approaches [29], we propose here a novel tool, named SMGen, designed to automatically generate synthetic biological networks codified as RBMs, which allow to obtain non-trivial dynamics. It is worth mentioning that we were mainly interested in the generation of RBMs suitable for the analysis and comparison of the performance of biochemical simulators. An example of such RMBs is a model composed of plausible reactions (e.g., transformation, production, and degradation reactions, which are common in real biochemical networks) leading to non-trivial dynamics. For instance, in the case of a dynamics that instantly exhausts all reactants, some of the most advanced integration algorithms are able to simulate such stable and/or flat dynamics in just one computation step, thus hampering the possibility of a fair comparison among the different simulation approaches.

In addition, SMGen overcomes the existing approaches and tools, which generally do not allow for using different probability distributions for the initialization of the species amounts and the kinetic constants. The possibility of exploiting different probability distributions for the initialization of these parameters is an important feature offered by SMGen, since the effort required for the simulation of a model can drastically vary according to its initial parameterization (i.e., species amounts and kinetic constants).

SMGen adheres to well-defined structural characteristics based on graph theory and linear-algebra properties; in particular, it exploits the definition of an undirected graph with a single connected component, which makes the whole generation process a computationally demanding task. To overcome this limitation, on the one hand, SMGen internally codifies all data structures by means of sparse matrices as well as ad-hoc structures specifically designed to avoid worthless values that would increase the running time required to generate RBMs. On the other hand, SMGen is able to drastically reduce the computational time by exploiting a main–worker paradigm used to distribute the overall generation process of RBMs onto multi-core CPUs. We show that SMGen can create, in less than 7 s, synthetic RBMs with hundreds of chemical species and molecular reactions, whose dynamics exhibit non trivial characteristics of real biochemical networks. Among the different features provided by SMGen, it allows for easily generating both symmetric and asymmetric RBMs: symmetric RBMs are composed of a number of species equal to the number of reactions, while, in asymmetric RBMs, the number of species can be lower than the number of reactions or vice-versa. From a computational point of view, the concept of symmetry is crucial in the analysis of complex networks to measure their information and entropy [30]. In addition, considering that every RBM can be converted into a corresponding system of coupled ordinary differential equations (ODEs), studying the symmetries of the system of ODEs can reveal the intrinsic properties of the system of interest. Ohlsson et al. pointed out that an alternative analysis of the system of ODEs can be carried out by considering the symmetries of the system solutions, aiming at formalizing the structures and behaviour of the underlying dynamics of biological systems [31]. Moreover, the possibility of evaluating GPU-powered simulators using symmetric and asymmetric RBMs is fundamental to understand their performance under different conditions. Indeed, a fair comparison would allow the user to select the best simulator based on characteristics of the RBM that has to be analysed. Thanks to its features and efficiency, SMGen was used to generate the synthetic RBMs necessary to realize a thorough comparison of the performance of different meta-heuristics in solving the PE problem of biochemical networks [3,5], which is one of the most common and difficult computational issues in systems biology. The outcome of such analyses showed that some well-known and widely used meta-heuristics, generally able to outperform all competitors in the optimization of benchmark functions, obtained poor performance when used to solve the PE problem. Moreover, SMGen was exploited to generate symmetric and asymmetric RBMs required for the in-depth analyses and comparisons of the computational performance of different biochemical simulators, and to highlight their peculiarities [19]. These analyses allowed for determining the best simulator to employ under specific conditions, such as the size of the RBM and the total number of simulations to perform. SMGen can also be used for the generation of biochemical networks to investigate the performance of approaches specifically designed to tackle other well-known and computationally demanding tasks in systems biology (e.g., SA and PSA [1,7]).

SMGen allows also for exporting the generated RBMs into the Systems Biology Markup Language (SBML) [32], Version 4 Level 2, and into the BioSimWare standard [33], which is used by different GPU-powered simulators. Thus, we designed and developed SMGen to be a unifying, user-friendly, and standalone tool freely accessible to the systems biology community. The RBMs can be easily generated by using the provided user-friendly graphical user interface (GUI), which was designed to help the users in setting all the parameters required to generate the desired RBMs.

The manuscript is structured as follows. Section 2 describes the mathematical formalism of RBMs, as well as the structural characteristics that must be complied to generate synthetic biological networks. In addition, we provide all the algorithms and details at the basis of SMGen. Section 3 shows the experimental results achieved by SMGen. Finally, a discussion and conclusive remarks are provided in Section 4.

2. Materials and Methods

2.1. Reaction-Based Models

An RBM is defined by specifying the set $\mathcal{S}=\{S_1,\dots ,S_N\}$ of N molecular species, and the set $\mathcal{R}=\{R_1,\dots ,R_M\}$ of M biochemical reactions that describe the interactions among the species appearing in $\mathcal{S}$. Each reaction $R_i$, with $i=1,\dots ,M$, is defined as:

$$R_i:\sum _{j=1}^N{a}_{ij}{S}_j\stackrel{k_i}{\to }\sum _{j=1}^N{b}_{ij}{S}_j,$$

(1)

where $a_{ij}$ and $b_{ij}\in \mathbb{N}$ are the stoichiometric coefficients, and $k_i\in {\mathbb{R}}^{+}$ is the kinetic constant associated with $R_i$. The stoichiometric coefficients specify how many molecules of species $S_j$, with $j=1,\dots ,N$, appear either as reactants or products in reaction $R_i$. Note that some species might not appear in a reaction, so that the corresponding stoichiometric coefficient will be equal to 0. The order of a reaction is equal to the total number of molecules (of the same or different species) that appear as reactants in that reaction.

Each RBM can be written in the compact matrix-vector form $\mathbf{A}\mathbf{S}\stackrel{\mathbf{K}}{\to }\mathbf{B}\mathbf{S}$, where $\mathbf{S}={[S_1\cdots S_N]}^{\top }$ is the N-dimensional column vector of the molecular species, $\mathbf{K}={[k_1\cdots k_M]}^{\top }$ is the M-dimensional column vector of the kinetic constants, while $\mathbf{A},\mathbf{B}\in {\mathbb{N}}^{M\times N}$ are the stoichiometric matrices, whose non-negative elements ${\left[A\right]}_{i,j}$ and ${\left[B\right]}_{i,j}$ correspond to the stoichiometric coefficients $a_{ij}$ and $b_{ij}$ of the reactants and products of the reactions, respectively.

Starting from an RBM and assuming the law of mass-action [34,35,36], the system of coupled ODEs corresponding to the RBM can be derived as follows:

$$\frac{d\mathbf{X}}{dt}={(\mathbf{B}-\mathbf{A})}^T[\mathbf{K}\circ {\mathbf{X}}^{\mathbf{A}}],$$

(2)

where each ODE describes the variation in time of a species’ concentration. In Equation (2), the N-dimensional vector $\mathbf{X}=[X_1\cdots X_N]$ represents the concentration values of species $S_1,\dots ,S_N$, while ${\mathbf{X}}^{\mathbf{A}}$ is the vector-matrix exponentiation form [34]; the symbol ∘ denotes the entry-by-entry matrix multiplication (Hadamard product).

2.2. SMGen

In order to generate synthetic models of biochemical networks, SMGen complies with specific structural characteristics that the RBMs have to satisfy, that is:

• System connectivity: a biochemical network can be represented as an undirected graph with a single connected component, where the nodes represent the molecular species, and the edges correspond to the species interactions. This representation easily allows for ensuring the system connectivity, a property that is strictly required to guarantee that each species $S_j\in \mathcal{S}$, with $j=1,\dots ,N$, will be involved in at least one reaction $R_i\in \mathcal{R}$, with $i=1,\dots ,M$.
  To be more precise, as a first step, an undirected graph with a single connected component is built. This undirected graph is randomly generated by using $N-1$ edges and by taking into account the maximum number of reactants and products, obtaining a connected biochemical reaction network. It is worth noting that a graph with a single connected component can be built if and only if the following condition is met:

  $$(ma{x}_{nu{m}_r}+ma{x}_{nu{m}_p})\times M>N,$$

  where $ma{x}_{nu{m}_r}$ and $ma{x}_{nu{m}_p}$ are the maximum number of reactants and products, respectively. As a second step, starting from the initial undirected graph, the stoichiometric matrices are generated. The stoichiometric matrices can be viewed and treated as a Petri net [37,38] that, in turn, can be considered as a bipartite graph. Then, the stoichiometric matrices are randomly updated by adding and removing connections among the species, always taking into consideration the maximum number of reactants and products. Note that the initial connections, which correspond to the edges of the initial undirected graph, are never removed to ensure that all the species are involved in at least one reaction, maintaining the whole biochemical network connected. We designed an algorithm (see Algorithm A1) that builds the graph, composed of a single connected component, with the minimum number of edges that are needed to connect all the nodes.
• Maximum number of reactants and products: for each reaction $R_i\in \mathcal{R}$, with $i=1,\dots ,M$, the number of reactants and the number of products cannot be arbitrarily large but has to be lower than or equal to a user-defined value (i.e., $ma{x}_{nu{m}_r}$ and $ma{x}_{nu{m}_p}$). Stated otherwise, the maximum order of the generated reactions should be fixed. SMGen does not explicitly account for conservation conditions during the generation of the RBMs; however, it can generate reactions that are akin to, e.g., protein modifications or conformational changes, thus resulting in two biochemical species whose sum is constant during the simulation.
• Linear independence: to ensure that each reaction $R_i$, with $i=1,\dots ,M$, is endowed with plausible characteristics of a real biochemical reaction, the vectors of the stoichiometric coefficients of the reactants and products involved in $R_i$ must be linearly independent. An example of an unrealistic reaction consists in increasing or decreasing the amount of a species starting from one molecule of the species itself, e.g., $R_i:S_j\stackrel{k_i}{\to }\alpha S_j$, with $\alpha >1$. Since the linear independence is evaluated between the reactants and the products of each reaction, it is related to the number of species and reactions. Thus, it could happen that when the number of species is much higher than the number of reactions it is not possible to build a graph with a single connected component. In such a case, duplicated reactions or linear-dependent vectors between reactants and products of some reactions will occur. On the contrary, it is always possible to build a graph, composed of a single connected component, when the number of reactions is much higher than the number of species.
• Reaction discreteness: each reaction $R_i$, with $i=1,\dots ,M$, must appear only once in the network; that is, duplicated reactions are not allowed.

SMGen is provided with a user-friendly GUI (see Figure 1) that allows the user to easily set up all the parameters required to generate the desired synthetic RBMs:

• the number of species N and the number of reactions M;
• the maximum number of reactants and products $ma{x}_{nu{m}_r}$ and $ma{x}_{nu{m}_p}$ that might appear in any reaction;
• the probability distribution ${\mathcal{D}}_s$ that is used to initialize the species amounts (to be chosen among uniform, normal, logarithmic, or log-normal distributions). Note that all species amounts are initialized using the same distribution probability;
• the minimum and maximum values $mi{n}_s$ and $ma{x}_s$ for the initial species amounts (to be specified either as number of molecules or concentrations);
• the probability distribution ${\mathcal{D}}_r$ that is used to set the values of the kinetic constants (to be chosen among uniform, normal, logarithmic, or log-normal distributions). Note that all kinetic constants are generated using the same distribution probability;
• the minimum and maximum values $mi{n}_r$ and $ma{x}_r$ for the kinetic constants;
• the total number of RBMs that the user wants to generate;
• the output format file to export the generated RBMs (i.e., BioSimWare [33] and SBML [32]);
• the mean and standard deviation values ${\mu }_s$ and ${\sigma }_s$ for the initial amounts—as well as the mean and standard deviation values ${\mu }_r$ and ${\sigma }_r$ for the kinetic constants—must also be provided if the normal or log-normal distributions are selected.

Figure 1. Graphical user interface of SMGen. The user can set all the parameters to generate the desired RBMs, i.e., number of species and reactions, maximum number of reactants and products, probability distribution for the initial amounts and kinetic constants, and the output format file (i.e., BioSimWare, SBML).

Figure 1. Graphical user interface of SMGen. The user can set all the parameters to generate the desired RBMs, i.e., number of species and reactions, maximum number of reactants and products, probability distribution for the initial amounts and kinetic constants, and the output format file (i.e., BioSimWare, SBML).

Figure 2 shows a high-level scheme of the proposed implementation of SMGen, which exploits the main–worker paradigm to speed up the generation of the RBMs [39]. The user can specify the number of processes P—otherwise automatically set to the minimum value 3—which are used as follows:

• Proc${}_1$ manages the GUI;
• Proc${}_2$ is the main process that orchestrates the computation;
• Proc${}_p$, with $p=3,\dots ,P$, are the worker processes.

Figure 2. Scheme of the main–worker implementation of SMGen. The main process (Proc${}_2$) orchestrates all the available workers (Proc${}_p$, with $p=3,\dots ,P$), which generate the RBMs in a distributed computing fashion.

Figure 2. Scheme of the main–worker implementation of SMGen. The main process (Proc${}_2$) orchestrates all the available workers (Proc${}_p$, with $p=3,\dots ,P$), which generate the RBMs in a distributed computing fashion.

The whole functioning of SMGen can be summarized as follows:

• the user interacts with the GUI, managed by Proc${}_1$, to fill in all the required values for the parameters necessary to create the RBMs;
• Proc${}_1$ sends the values of all parameters to the main process (Proc${}_2$), which allocates the resources and distributes the work to the workers (Proc${}_p$, with $p=3,\dots ,P$);
• each worker (Proc${}_p$, with $p=3,\dots ,P$) generates an RBM. As soon as a worker terminates its execution, it communicates to the main process that the RBM has been created. If necessary, the main process assigns the generation of other RBMs to idle workers. When all the required RBMs are obtained, the workers enter in the death state, while the main process waits for further instructions from Proc${}_1$.

The workflow of each worker consists in 9 different phases, in which a specific algorithm is executed (see Figure 3).

The pseudo-code reported in Algorithm 1 briefly summarizes all the steps required to generate a single RBM; the pseudo-code of the procedures invoked within Algorithm 1 are reported in Appendix A. For the sake of clarity, Table 1 lists the symbols used in the following description and in the pseudo-codes.

Algorithm 1 SMGen: workflow of a single worker execution.

1:   function generator($M,N,ma{x}_{nu{m}_p},ma{x}_{nu{m}_r},{\mathcal{D}}_s,{\mathcal{D}}_r,mi{n}_s,ma{x}_s,mi{n}_r,ma{x}_r,{\mu }_s,{\sigma }_s,{\mu }_r,{\sigma }_r$) 2:   ## Algorithm A1 3:   $\mathbf{G}\leftarrow$ GRAPH_GEN(N) 4:   ## Algorithm A2 5:   $\mathbf{A},\mathbf{B}\leftarrow$ STOICH_MATRICES_GEN($M,N,\mathbf{G}$) 6:   $AIJ[·],BIJ[·]\leftarrow \left[\right]$                                          $▹$ 7:   for $i=1$ to M do                                            $▹$ 8:    for $j=1$ to N do                                              $▹$ 9:       if $\mathbf{A}[i,j]==1$ then                                             $▹$ 10:        $AIJ\leftarrow AIJ\odot \langle i,j\rangle$                                           $▹$ 11:       if  $\mathbf{B}[i,j]==1$ then                                       $▹$ 12:        $BIJ\leftarrow BIJ\odot \langle i,j\rangle$                                     $▹$ 13:   ## Algorithm A3 14:   $\mathbf{A},\mathbf{B}\leftarrow$ STOICH_COEFFICIENTS_GEN($\mathbf{A},\mathbf{B},M,N,ma{x}_{nu{m}_r},ma{x}_{nu{m}_p},AIJ,BIJ$) 15:   ## Algorithm A4 16:   $er{r}_{LinDep}\leftarrow$LINEAR_INDEPENDENCE_MATRIX($\mathbf{A},\mathbf{B},M$) 17:   ## Algorithm A5 18:   $er{r}_{Repeat}\leftarrow$UNIQUE_REACTIONS($\mathbf{A},\mathbf{B},M$) 19:   while $er{r}_{LinDep}\wedge er{r}_{Repeat}$ are not empty do 20:     $row{s}_{Err}\leftarrow unique(er{r}_{LinDep}\odot er{r}_{Repeat})$ 21:     ## Algorithm A6 22:     $\mathbf{A},\mathbf{B}\leftarrow$ CORRECTION_REACTIONS($\mathbf{A},\mathbf{B},row{s}_{Err},AIJ,BIJ,ma{x}_{nu{m}_r},ma{x}_{nu{m}_p}$) 23:     ## Algorithm A7 24:     $er{r}_{LinDep}\leftarrow$ LINEAR_INDEPENDENCE_REACTION($\mathbf{A},\mathbf{B},row{s}_{Err}$) 25:     ## Algorithm A5 26:     $er{r}_{Repeat}\leftarrow$UNIQUE_REACTIONS($\mathbf{A},\mathbf{B},M$) 27:   ## Algorithm A8 28:   ${\mathbf{M}}_{\mathbf{0}}\leftarrow$ AMOUNTS_GEN($N,{\mathcal{D}}_s,mi{n}_s,ma{x}_s,{\mu }_s,{\sigma }_s$) 29:   ## Algorithm A9 30:   $\mathbf{K}\leftarrow$ KINETIC_CONSTANTS_GEN($M,{\mathcal{D}}_r,mi{n}_r,ma{x}_r,{\mu }_r,{\sigma }_r$) $▹\to$ the instructions shown in lines 6–12 are required to build the structure of the initial graph of the reactions.

The steps performed by each worker to generate an RBM are the following:

• Given the parameters provided by the user, the graph representing the species and their interactions is randomly initialized (line 3 of Algorithm 1; see Algorithm A1).
• The adjacency matrix of the graph generated in Step 1 is converted into the stoichiometric matrices $\mathbf{A}$ and $\mathbf{B}$ (line 5 of Algorithm 1; see Algorithm A2). Note that the instructions in lines 6–17 of Algorithm 1 are required to build the data structure of the initial graph, which is then modified.
• The stoichiometric coefficients are randomly generated (line 19 of Algorithm 1; see Algorithm A3).
• For each reaction $R_i$, with $i=1,\dots ,M$, the linear independence between the reactants and products is verified (line 21 of Algorithm 1; see Algorithm A4).
• The uniqueness of each reaction in the RBM is verified (line 23 of Algorithm 1; see Algorithm A5).
• Any error in the RBM identified in the previous steps is corrected (line 27 of Algorithm 1; see Algorithm A6); the linear independence and the uniqueness of the reactions in the modified RBM are iteratively verified (lines 29 and 31 of Algorithm 1; see Algorithms A5 and A7, respectively).
• The initial amounts of the species are generated according to the chosen probability distribution (line 33 of Algorithm 1; see Algorithm A8). If a species appears only as a reactant in the whole RBM, its amount is set to remain unaltered. The rationale behind this is double: on the one hand, we avoid the possibility of creating reactions that could be applied at most once, which is a highly improbable situation in biological systems; on the other hand, we mimic the non-limiting availability of some biochemical resources, for instance, it might be used to reproduce the execution of in vitro experiments where some species are continually introduced in the systems to keep their amount constant [40].
• The kinetic constants of the reactions are generated according to the chosen probability distribution (line 35 of Algorithm 1; see Algorithm A9).

SMGen was developed using the Python programming language and exploiting mpi4py [41], which provides bindings of the message passing interface (MPI) specifications for Python to leverage multi-core CPUs [42]. The open-source code of SMGen is available on GitLab (https://gitlab.com/sgr34/smgen, accessed on 2 December 2021) under the GNU GPL-3 license. In addition, it can be easily installed using the Python package installer pip (https://pypi.org/project/smgenerator, accessed on 2 December 2021).

Table 1. List of symbols used in the pseudo-code of algorithms at the basis of SMGen.

Symbol	Description
M	Number of reactions composing the RBM
N	Number of species involved in the RBM
$ma{x}_{nu{m}_r}$	Maximum number of the reactants
$ma{x}_{nu{m}_p}$	Maximum number of the products
${\mathbf{M}}_{\mathbf{0}}$	Array of the initial amounts
$\mathbf{K}$	Array of the kinetic constants
$\mathbf{A}$	Stoichiometric matrix of the reagents
$\mathbf{B}$	Stoichiometric matrix of the products
$\mathbf{G}$	Adjacency matrix of the graph of the reactions
${\mathcal{D}}_s$	Probability distribution for the initial amounts
$mi{n}_s$	Minimum value of the initial amounts
$ma{x}_s$	Maximum value of the initial amounts
${\mu }_s$	Mean of the normal and log-normal distributions for the initial amounts
${\sigma }_s$	Standard deviation of the normal and log-normal distributions for the initial amounts
${\mathcal{D}}_r$	Probability distribution for the kinetic constants
$mi{n}_r$	Minimum value of the kinetic constants
$ma{x}_r$	Maximum value of the kinetic constants
${\mu }_r$	Mean of the normal and log-normal distributions for the initial amounts
${\sigma }_r$	Standard deviation of the normal and log-normal distributions for the kinetic constants amounts
⊙	The concatenation operator

Table 1. List of symbols used in the pseudo-code of algorithms at the basis of SMGen.

Symbol	Description
M	Number of reactions composing the RBM
N	Number of species involved in the RBM
$ma{x}_{nu{m}_r}$	Maximum number of the reactants
$ma{x}_{nu{m}_p}$	Maximum number of the products
${\mathbf{M}}_{\mathbf{0}}$	Array of the initial amounts
$\mathbf{K}$	Array of the kinetic constants
$\mathbf{A}$	Stoichiometric matrix of the reagents
$\mathbf{B}$	Stoichiometric matrix of the products
$\mathbf{G}$	Adjacency matrix of the graph of the reactions
${\mathcal{D}}_s$	Probability distribution for the initial amounts
$mi{n}_s$	Minimum value of the initial amounts
$ma{x}_s$	Maximum value of the initial amounts
${\mu }_s$	Mean of the normal and log-normal distributions for the initial amounts
${\sigma }_s$	Standard deviation of the normal and log-normal distributions for the initial amounts
${\mathcal{D}}_r$	Probability distribution for the kinetic constants
$mi{n}_r$	Minimum value of the kinetic constants
$ma{x}_r$	Maximum value of the kinetic constants
${\mu }_r$	Mean of the normal and log-normal distributions for the initial amounts
${\sigma }_r$	Standard deviation of the normal and log-normal distributions for the kinetic constants amounts
⊙	The concatenation operator

3. Results

We analysed the performance of SMGen regarding both its capability of creating RBMs characterized by non-trivial dynamics and the computational time required to generate sets of RBMs of increasing size. All tests were executed on a workstation equipped with an Intel Core i7-8750H CPU (clock $4.1$ GHz), 16 GB of RAM, and a Samsung 970 EVO solid-state drive NVMe PCIe (up to 3400 MB/s and 1500 MB/s read and write speeds, respectively), running on Ubuntu 20.04 LTS.

As a first batch of tests, we generated 100 synthetic RBMs characterized by a limited number of reactions and species (4 and 5, respectively), and we analysed their characteristics and dynamics. We set to 3 both the maximum number of reactants $ma{x}_{nu{m}_r}$ and products $ma{x}_{nu{m}_p}$. We sampled the initial amounts of species from a normal distribution with mean ${\mu }_s=5$ and standard deviation ${\sigma }_s=5$, considering a minimum value $mi{n}_s=0$ and a maximum value $ma{x}_s=10$. The kinetic constants were sampled from a logarithmic distribution with minimum value $mi{n}_r={10}^{-16}$ and maximum value $ma{x}_r=10$.

Table 2. List of the reactions of an RBM with 4 reactions and 5 species generated by SMGen.

No.	Reagents	Products	Constant
$R_1$	$X_0+X_4$	$X_3$	$4.295\times {10}^{-5}$
$R_2$	$X_4$	$X_1+2X_2$	$2.207\times {10}^{-2}$
$R_3$	$X_4$	$X_2+X_4$	$7.070\times {10}^{-4}$
$R_4$	$X_1+X_4$	$X_2+X_3$	$4.613\times {10}^{-2}$

shows the list of reactions along with the kinetic constants of one of these 100 synthetic RBMs. Since the species $X_0$ appears only as a reactant, its amount will be kept constant during the simulation. The initial molecular amounts of all species—given as the number of molecules—are listed in

Table 3. Initial molecular amounts of the RBM generated by SMGen shown in Table 2.

Species	Initial Amount
$X_0$	4
$X_1$	8
$X_2$	7
$X_3$	8
$X_4$	1

. This small RBM includes the basic “cascade of reactions” structure typically observed in signalling pathways, starting from the source represented by species $X_0$ and $X_4$, toward species $X_2$ and $X_3$.

We simulated the dynamics of this RBM for 50 time steps (arbitrary units) using FiCoS [19], and the achieved dynamics are shown in Figure 4. These plots evidence that, although the RBM was randomly generated by SMGen, the simulated behaviour is non-trivial (e.g., the reactants were not instantly exhausted, resulting in flat dynamics). The capability of generating synthetic RBMs that exhibit non-trivial dynamics is fundamental to perform in-depth computational analyses and comparisons among any existing and novel simulators. Indeed, in the case of stable or flat dynamics, or when the overall behaviour of the network is extremely fast and all the reactants are immediately depleted, some of the most advanced integration algorithms can simulate the emergent dynamics in just one computation step [19]. In such a case, the computational performance of the simulation tools is only partially assessed, thus hindering a fair comparison among the tools.

As a second batch of tests, we evaluated the computational performance of SMGen exploiting the main–worker paradigm running on 4 distinct cores of the CPU. First, we considered the generation of symmetric RBMs with an increasing number of species and reactions (i.e., $M=N=2^x$, with $x=2,\dots ,9$). The initial amounts and kinetic constants were randomly sampled from a uniform distribution with minimum values $mi{n}_s=mi{n}_r=0$ and maximum values $ma{x}_s=ma{x}_r=10$. We also varied the maximum numbers of reactants and products considering the set of values $\{2,3,4\}$, and setting $ma{x}_{nu{m}_r}=ma{x}_{nu{m}_p}$. For each of the resulting 24 parameters combinations, we created 100 RBMs to collect statistically sound results about the performance of SMGen. As described in Section 2, two kinds of error can occur during the generation of an RBM: a linear dependence between reactants and products, and duplicated reactions. Since the correction of these errors is one of the most time-consuming phases of SMGen, we separately measured the generation time, which indicates the running time spent by SMGen to generate an RBM, and the saving time, which refers to the writing operations on the solid-state drive. Figure 5 shows the average running time required by SMGen to generate and save an RBM. As expected, both the generation and the saving time increase along with the number of species and reactions of the RBM. Moreover, we observed that the maximum number of reactants and products have a slight impact on both the generation and the saving time; in most of the cases, increasing these values resulted in a higher running time.

Finally, we exploited SMGen for the creation of asymmetric RBMs to evaluate how a different number of species and reactions affects the running time. As in the case of symmetric RBMs, we measured both the generation time and the saving time. The asymmetric RBMs were created as follows:

• we set the number of species $N\in \{4,8,16,32,64\}$, and then we varied the number of reactions $M\in \{2N,4N,8N\}$;
• we set the number of reactions $M\in \{4,8,16,32,64\}$, and then we varied the number of species $N\in \{2M,4M,8M\}$;
• we varied both the maximum numbers of reactants $ma{x}_{nu{m}_r}$ and products $ma{x}_{nu{m}_p}$ in $\{2,3,4\}$.

In such a way, we obtained a total of 90 different combinations of the parameters to be tested (i.e., number of species, number of reactions, and maximum number of reactants and products); as in the previous tests, for each combination we generated 100 RBMs to collect statistically sound results. Figure 6 shows the average running time required to create RBMs with dimensions $N\times M$, highlighting once again that both the generation time and the saving time increase along with the size of the RBMs. As in the case of symmetric RBMs, we observed the same effect due to the maximum number of reactants and products allowed in the reactions. As expected, when there are more reactions than species (bottom panel in Figure 6), the generation times is higher than the opposite situation (top panel in Figure 6). This circumstance is due to the potential higher number of errors that SMGen has to identify and correct. Indeed, when $M\gg N$, the probability that repeated reactions are randomly generated is higher than the case when $N\gg M$, because the number of admissible reactions strictly depends on the number of species.

4. Conclusions

In this work we presented SMGen, a generator of synthetic RBMs displaying the characteristics of real biochemical networks, which can be exploited to create benchmarks for the evaluation of novel and existing simulators. In particular, SMGen is suitable for GPU-based simulators, since their performance can drastically change with the number of chemical species and reactions composing an RBM. Indeed, given the system of coupled ODEs corresponding to a RBM, the resolution of the system of ODEs can be performed in a parallel fashion, where each ODE is resolved by a thread. Since each ODE is related to a specific chemical species, the higher the number of species the higher the parallelization, which increases the computational performance of the simulator. On the contrary, considering that the number of the reactions composing the biological system is roughly related to the length of each ODE, in terms of the mathematical complexity, the higher the number of reactions the higher the number of operations that must be performed by each thread, leading to a higher running time [18,19].

SMGen was developed in Python and was designed to be a unifying, user-friendly, and standalone tool. In addition, SMGen exploits the main–worker paradigm to speed up the generation of RBMs; this was implemented using the mpi4py [41] library, where the first process manages the GUI, the second one is the main process, and all the other processes are the workers that generate the RBMs in a distributed computing fashion. Thanks to the GUI of SMGen, the user can easily set up all the parameters characterizing the required RBMs, e.g., the number of species and reactions, the maximum number of reactants and products per reaction, the probability distributions (uniform, normal, logarithmic, and log-normal) to generate the initial amounts of the species and the values of the kinetic constants associated with the reactions, and the output file format to save the RBMs. It is noteworthy that the performance of SMGen is not affected by the choice of different distributions for the sampling of the species amounts and kinetic constants. On the contrary, different distributions can drastically affect the dynamics of the model (see, e.g., the Brusselator model [38,43]). The analysis of the features of the generated models, according to different distributions, will be addressed in a future work.

We assessed the capabilities of SMGen for the creation of RBMs characterized by non-trivial behaviour, and we presented an example of a synthetic RBM together with the simulated dynamics. We also tested the computational performance of SMGen by generating batches of symmetric and asymmetric RBMs of increasing size, showing the impact of the number of reactions and species, and of the number of reactants and products per reaction, on the generation times. We observed that when the number of reactions is higher than the number of species, SMGen generally identifies and corrects high numbers of errors during the creation process of the RBMs, a circumstance that inevitably increases the overall running time.

As a future extension of this work, we plan to develop an application programming interface (API), so that SMGen can be seamlessly integrated into other processing pipelines, tools, and simulators. We are also developing a well-documented command-line interface, which will be released together with the API, to increase the user experience as well as the integrability with the existing approaches. Another feature that could be introduced is a module for the visualization of the networks, especially those that do not consist in a huge number of entities; indeed, the visualization of networks composed of thousands of species and reactions might not provide relevant information to the user. We will also introduce a new feature specifically developed to generate feedback loops in synthetic RBMs, exploiting the theory of Petri nets [37,38]. Feedback loops are fundamental elements of biological processes that lead to the establishment of oscillatory regimes and non-linear dynamics [22]. Moreover, we plan to develop a function to rescale the kinetic constants by taking into account the number of reactants involved in the reactions, as the order of magnitude and the value of each kinetic constant are related to the number of reactants composing the corresponding reaction. In the current version of SMGen, we assumed that all reactions follow the law of mass-action; we will implement additional kinetics (e.g., Michaelis–Menten and Hill kinetics [44,45]) in the future.

SMGen relies on graph theory and linear-algebra properties to comply with specific structural characteristics that are essential to create synthetic but real RBMs. Nevertheless, the current version of SMGen does not exploit network graphlets and motifs, which characterize families of real biological networks. Graphlets are induced sub-graphs appearing at any frequency because they are independent of the network’s null model (i.e., a random graph model defined by a probability distribution), while network motifs are repeated sub-graphs appearing with a frequency higher than in random graphs and depending on the network’s null model [46]. Graphlets have been used to analyse local network structures and to cluster different network types [46]. Probabilistic graphlets have also been developed to analyse the local wiring patterns of probabilistic networks. When applied to study biological networks, probabilistic graphlets resulted in a robust tool, which was able to capture the information underlying biological networks thanks to the capabilities of managing the low-signal topology information [47]. Graphlets can also be used to measure the structural similarity among large networks, calculating the graphlet degree distribution [48]. Motifs can be used to study autoregulation, single-input modules, dense overlapping regulons, and feedback loops as well as to reveal answers to many important biological questions [49,50]. The frequencies of the motifs were also exploited as classifiers for the selection of the network models [51]. Studying the motifs is also fundamental to understand the stability and robustness of the biological networks in response to small perturbations [52]. As a matter of fact, Prill et al. showed that the stability and robustness of a network is strictly related to the relative abundance of the motifs [52]. This result suggests that the structural organization of a biological network can be highly related to the dynamic properties of the small network motifs. We plan to extend SMGen to incorporate network graphlets and motifs during the generation of the RBMs, to obtain synthetic models with an improved biological significance.

In addition, we will modify the random generation of the kinetic constants to take into account the fact that different biological processes can operate on different time scales. In order to introduce this modification, as a first step, the reactions will be grouped into different families based on the biological process that they describe (e.g., protein or mRNA degradation, transcription rates, translation rates). Then, for each family of reactions, a different probability distribution will be used to sample the kinetic constants to reflect the time scale of the described biological process. We are also investigating the possibility of generating models characterized by specific emergent dynamical aspects (e.g., stable oscillatory regimes, state switching, and multi-stability [16]). We derived a set of heuristics to detect these conditions from the time-series, based on the combination of SMGen with GPU-powered stochastic simulators [16]. Still, regardless of the acceleration used, robustly detecting these phenomena is computationally challenging because some parameterizations might lead to very stiff models, which, in turn, might require a very long running time. Thus, we are investigating alternative approaches to guide the generation of synthetic models having such features. In particular, we are considering the possibility of providing a set of “functional modules”, i.e, a group of already parameterized reactions implementing specific behaviours. These modules could be coerced into the synthetic models generated by SMGen, possibly leading to the emergence of the desired phenomena. Finally, we plan to include an initial check of the parameter values set by the user, based on some heuristics, to verify whether the RBMs can be actually generated as requested. This initial step will allow for avoiding worthless calculations and to suggest useful modifications of parameters to the user.