Model Reduction for Kinetic Models of Biological Systems

Abstract

High dimensionality continues to be a challenge in computational systems biology. The kinetic models of many phenomena of interest are high-dimensional and complex, resulting in large computational effort in the simulation. Model order reduction (MOR) is a mathematical technique that is used to reduce the computational complexity of high-dimensional systems by approximation with lower dimensional systems, while retaining the important information and properties of the full order system. Proper orthogonal decomposition (POD) is a method based on Galerkin projection that can be used for reducing the model order. POD is considered an optimal linear approach since it obtains the minimum squared distance between the original model and its reduced representation. However, POD may represent a restriction for nonlinear systems. By applying the POD method for nonlinear systems, the complexity to solve the nonlinear term still remains that of the full order model. To overcome the complexity for nonlinear terms in the dynamical system, an approach called the discrete empirical interpolation method (DEIM) can be used. In this paper, we discuss model reduction by POD and DEIM to reduce the order of kinetic models of biological systems and illustrate the approaches on some examples. Additional computational costs for setting up the reduced order system pay off for large-scale systems. In general, a reduced model should not be expected to yield good approximations if different initial conditions are used from that used to produce the reduced order model. We used the POD method of a kinetic model with different initial conditions to compute the reduced model. This reduced order model is able to predict the full order model for a variety of different initial conditions.

1. Introduction

In biological systems, kinetic models of biochemical networks are necessary for predicting and optimizing the behavior of cells in culture. Most of these models are high-dimensional and involve a large number of reactions and species and reaction rate constants of widely different orders of magnitude [1]. Thus, model order reduction (MOR) is considered to be a vital topic and essential for an efficient numerical simulation. In general, model order reduction is a mathematical theory for reducing the computational complexity of large-scale dynamical systems via finding a low-dimensional approximation while preserving the most important information of the full order system. The computational complexity of a method is a measure of time or storage space that the method requires to solve a given task. In this paper, we reduce the time cost of the simulation of dynamical systems using MOR methods, as we will discuss later; see Section 3. Moreover, MOR methods of a higher order dimensional system produce a lower order dimensional system, which is considered more efficient for reducing the computational effort in mathematical processes, e.g., an optimization process.

The model reduction method has been discussed in mathematical modeling of biological systems. Here, Michaelis–Menten kinetics [2,3], whose derivation is based on a quasi-steady state assumption [4,5], is considered the famous example of the model reduction theory of enzymatic reactions. Many other model reduction methods [6] have been discussed that are based on different ways of analyzing the system. Sensitivity analysis [7,8] is based on eliminating parameters that are noticed to be the least sensitive in affecting the model. The lumping method was initially presented in the 1960s by Wei and Kuo [9,10]. The idea behind the lumping method is to remove a group of state variables from the system and replace them with a new dynamical variable (lumped), which is related to the original system by lumping functions [11,12]. Time scale analysis [13] is the most recent model reduction approach that is used in biological systems. It assumes two time scales, fast and slow, whereby the fast one converges to a quasi-steady state. However, in some cases, the differences in the time scales do not suffice for obtaining a reduced model that behaves in the same way as the original model [14]. Thus, the required conditions of Tikhonov’s theorem (see [15,16]) should be satisfied to apply the model order reduction method. In [17], the model reduction of a small single metabolic-genetic network was studied using the time scale separation. The model reduction of a two-stage anaerobic digestion process model was also studied in [18], and the model reduction of RNA polymerase was studied in [19].

The conditions that have to hold for the applicability of Tikhonov’s theorem are not trivial, especially for a large-scale dynamical system. Therefore, we aim to apply another popular approach for model reduction: the proper orthogonal decomposition (POD), a method that aims at obtaining low-dimensional approximate descriptions of high-dimensional systems while preserving important information. The POD method, also known as Karhunen–Loève Expansion (KLE) [20,21], is based on the idea of projecting the full order system onto a lower dimensional subspace while capturing the main characteristics of the system. Within the POD method, the key idea is to use singular value decomposition (SVD) [22,23,24] to calculate the basis for the low-dimensional subspace. The SVD is considered to be one of the most important matrix factorizations in the field of data science, since it exists for any matrix and can be used for approximating high-dimensional data by low-dimensional data in terms of dominant patterns.

A drawback of the POD method is that for time-dependent and/or parameterized nonlinear systems, the computational complexity for evaluating the nonlinear term in the reduced order model remains that of the original problem, although the dimension of the system is reduced. The discrete empirical interpolation method (DEIM) was proposed in [25] for high-dimensional nonlinear ODE models and is able to greatly improve the dimension reduction efficiency of the POD method.

In this paper, we discuss the model order reduction techniques by POD and DEIM for kinetic models of biological systems. From a practical point of view, the POD and DEIM methods are more advantageous for large-scale dynamical systems than time scale separation techniques, since we do not have to take care of satisfying the conditions required, e.g., for Tikhonov’s theorem.

The methods are applied to a kinetic model of a metabolic-genetic network that was introduced in [26]. In addition, we apply the approaches to different models of biological systems from the BioModels database (http://identifiers.org/biomodels.db), and we compare the time costs of the simulation for the original and the reduced order model in different cases. The MOR methods depend on a snapshot matrix generated from numerical simulations of the dynamical system with certain initial conditions. In general, a reduced model should not be expected to yield good approximations if different initial conditions are used from those used to produce the reduced order model [27].

However, an important characteristic of a kinetic model in systems biology is that it can predict different scenarios of behavior for the system that can be stimulated by prescribing different initial conditions. Here, we use the POD approach to compute a reduced order model of a kinetic model for different initial conditions of the dynamical system. This reduced order model is able to predict the full order model for a variety of different initial conditions. Using different initial conditions in the time scale separation technique will usually fail, since the algebraic equations of the fast variables may not be solvable if inconsistent initial values are prescribed; see [17].

The remainder of this paper is organized in the following manner. We discuss the model reduction of a dynamical system using POD and DEIM in Section 2. We then apply the approaches to a simplified metabolic-genetic network and to different kinetic models from the BioModels database in Section 3. We study the POD method of a kinetic model with different initial conditions in Section 4. Finally, we end with some concluding remarks in Section 5.

2. Proper Orthogonal Decomposition for Differential Equations

We consider a system of ordinary differential equations that is given by:

$${\displaystyle \frac{d}{dt}}x\left(t\right)=f(x\left(t\right),\theta ),x\left(0\right)=x_0,\mathrm{for}t\in [0,T],$$

(1)

where $x\left(t\right)$ is the m-dimensional state vector of the model and f is a nonlinear function mapping from $D_x\subset {\mathbb{R}}^m\times D_{\theta }$ to the m-dimensional space ${\mathbb{R}}^m$ depending on parameters $\theta \in D_{\theta }\subset {\mathbb{R}}^{n_{\theta }}$. Moreover, $n_{\theta }$ denotes the number of parameters, and $x_0\in {\mathbb{R}}^m$ is a given initial value.

The proper orthogonal decomposition (POD) method is a projection-based order reduction technique that aims at constructing a reduced order system of order $k\ll m$ approximating the original system (1) from a subspace that is spanned by a reduced basis of dimension k in ${\mathbb{R}}^m$ using Galerkin projection. Let $V_k\in {\mathbb{R}}^{m\times k}$ denote a matrix whose orthonormal columns form the reduced POD basis, then the reduced system of (1) is obtained by replacing $x\left(t\right)$ in (1) by $V_k\tilde{x}\left(t\right)$, $\tilde{x}\left(t\right)\in {\mathbb{R}}^k$ and projecting the system (1) onto $V_k$ to get:

$$\frac{d}{dt}\tilde{x}\left(t\right)=V_k^{T}f(V_k\tilde{x}\left(t\right),\theta ),\tilde{x}\left(0\right)=V_k^T{x}_0,\mathrm{for}t\in [0,T].$$

(2)

In the POD approach, a reduced basis is constructed from a truncated singular value decomposition (SVD) of a matrix of sample trajectory vectors (snapshots), assuming that the samples are on or near the attracting low-dimensional manifold that represents the solution space.

We introduce the trajectory snapshot matrix $\mathbb{X}=[x_1,x_2,\cdots ,x_{n_s}]\in {\mathbb{R}}^{m\times n_s},$ consisting of a time series of data (samples of trajectories) and assume that $\mathrm{rank}\mathbb{X}=r\le min(m,n_s)$. This snapshot matrix can be obtained via a numerical solution of the dynamical system or from measured datasets. The SVD guarantees the existence of real non-negative numbers ${\sigma }_1\ge {\sigma }_2\ge \cdots \ge {\sigma }_r>0$ and orthogonal matrices $V=[v_1,\cdots ,v_m]\in {\mathbb{R}}^{m\times m}$ with columns ${\left\{v_i\right\}}_{i=1}^m$ and $W=[w_1,\cdots ,w_{n_s}]\in {\mathbb{R}}^{n_s\times n_s}$ with columns ${\left\{w_i\right\}}_{i=1}^{n_s}$ such that:

$$\begin{array}{c}\hfill V^T\mathbb{X}W=\left[\begin{array}{cc}D& 0\\ 0& 0\end{array}\right]=:\mathsf{\Sigma }\in {\mathbb{R}}^{m\times n_s}\end{array}$$

(3)

where $D=diag({\sigma }_1,\cdots ,{\sigma }_r)\in {\mathbb{R}}^{r\times r}$ and the zeros denote matrices of appropriate dimensions. Similarly, we have:

$$\mathbb{X}=V\mathsf{\Sigma }{W}^T.$$

Then, the POD basis is given by ${\left\{v_i\right\}}_{i=1}^k$. We set $V_k=[v_1,\cdots ,v_k]\in {\mathbb{R}}^{m\times k}$, and the linear span of the vectors $\{v_1,\cdots ,v_k\}$ determines the best rank-k approximation of $Range\left(\mathbb{X}\right)$ in the sense that is minimizes the two-norm of the approximation error:

$$\sum _{j=1}^{n_s}{∥x_j-V_k{V}_k^T{x}_j∥}_2^2=\sum _{i=k+1}^r{\sigma }_i^2;$$

see [28]. In this truncation of the SVD, we try to approximate the matrix $V\mathsf{\Sigma }{W}^T$ with k chosen as small as possible, but as large as necessary to obtain a good approximation. The strong decay of singular values allows a good representation with only a few modes. If the decay of singular values is gradual, one is forced to use a high number of modes to get a good approximation.

The general philosophy in model order reduction is to cut off singular values with ${\sigma }_i<\epsilon$, where $\epsilon$ is chosen such that a (much) smaller number of basis elements $k\ll m$ is sufficient to capture the main features of the solution of (1). The goal is to choose k small enough, while the relative information content [29] of the basis for the k-dimensional subspace, defined by:

$$I\left(k\right)=\frac{{\sum }_{i=1}^k{\sigma }_i^2}{{\sum }_{i=1}^r{\sigma }_i^2},$$

(4)

is near one. If the k-dimensional subspace should contain a percentage p of the information contained in the full-dimensional space ${\mathbb{R}}^n$, then one should choose k such that:

$$k=\mathrm{argmin}\left\{I\left(k\right)|I\left(k\right)\ge \frac{p}{100}\right\};$$

see [29].

A drawback of the POD method is the computational complexity for time-dependent and/or parameterized nonlinear systems. Although the dimension of the system is reduced in the sense that far fewer variables are present in the reduced order system (2), the complexity of evaluating the nonlinear term remains that of the original problem. The nonlinear term:

$$\tilde{N}\left(\tilde{x}\right):=\underset{k\times m}{\underbrace{V_k^T}}\underset{m\times 1}{\underbrace{f(V_k\tilde{x}\left(t\right),\theta )}}$$

(5)

in the reduced system (2) has a computational complexity that depends on m, the dimension of the original full-order system (1), and in particular, it requires a full evaluation of the nonlinear function f at the m-dimensional vector $V_k\tilde{x}\left(t\right)$, such that solving this system is still as costly as solving the original system.

The discrete empirical interpolation method (DEIM) was proposed in [25] for high-dimensional nonlinear ODE models to improve greatly the dimension reduction efficiency of POD with Galerkin projection by approximating the nonlinear function by combining projection with interpolation.

The nonlinear function $f(V_k\tilde{x}\left(t\right),\theta )$ is approximated by projecting it onto a subspace spanned by a POD basis of dimension $\ell \ll m$. This POD basis $\{u_1,\cdots ,u_{\ell }\}$ for the nonlinear function is constructed by applying the POD to the nonlinear snapshot matrix $\mathbb{F}=[f_1,\cdots ,f_{n_s}]\in {\mathbb{R}}^{m\times n_s}$ obtained from the original full order model (1), where $f_j=f(x_j,\theta )$ and $x_j\approx x\left(t_j\right)$ with $k,\ell \le min\{m,n_s\}$. The values in $\mathbb{F}$ are already needed to generate the snapshot matrix $\mathbb{X}$, and thus, it requires no additional cost other than the truncated SVD to obtain $U_{\ell }=[u_1,\cdots ,u_{\ell }]\in {\mathbb{R}}^{m\times \ell }$. Similar as above, the two-norm error for the approximation of the nonlinear function is given by:

$$\sum _{j=1}^{n_s}{∥f_j-U_{\ell }{U}_{\ell }^T{f}_j∥}_2^2=\sum _{i=\ell +1}^{r_s}{\rho }_i^2,$$

where $f_j=f(x\left(t_j\right),\theta )$, $t_j\in [0,T]$, $r_s:=rank\left(\mathbb{F}\right)$, and ${\rho }_i$ denotes the singular values of $\mathbb{F}$; again, see [28].

The corresponding POD-DEIM reduced system is designed by applying Galerkin projection on the space spanned by the columns of the POD basis matrix $V_k$ and then applying the DEIM approximation to the nonlinear function using interpolation projection onto the columns space of the POD basis matrix $U_{\ell }$. The resulting reduced system is then given by:

$$\frac{d}{dt}\tilde{x}\left(t\right)=V_k^T\mathbb{P}f(V_k\tilde{x}\left(t\right),\theta ),\tilde{x}\left(0\right)=V_k^T{x}_0,\mathrm{for}t\in [0,T],$$

(6)

where $\mathbb{P}:=U_{\ell }{\left(P^T{U}_{\ell }\right)}^{-1}{P}^T\in {\mathbb{R}}^{m\times \ell }$ and $P\in {\mathbb{R}}^{m\times \ell }$ is a matrix whose columns come from some selected columns of the identity matrix corresponding to the so-called DEIM indices. In more detail, the nonlinear function in (2) approximated by DEIM can be written as:

$$f(V_k\tilde{x}\left(t\right),\theta )\approx \mathbb{P}f\left(V_k\tilde{x}\left(t\right)\right)=:\widehat{f}\left(V_k\tilde{x}\left(t\right)\right),$$

such that the nonlinear term (5) can be approximated by:

$$\tilde{N}\left(\tilde{x}\right)\approx \underset{k\times \ell }{\underbrace{V_k^T{U}_{\ell }{\left(P^T{U}_{\ell }\right)}^{-1}}}\underset{\ell \times 1}{\underbrace{P^{T}f\left(V_k\tilde{x}\left(t\right)\right)}}.$$

The first term does not depend on t and can be precomputed before solving the system of ODEs. The interpolation indices $\{p_1,\cdots ,p_{\ell }\}$ used in the construction of the matrix $P=[e_{p_1},\cdots ,e_{p_{\ell }}]\in {\mathbb{R}}^{m\times \ell },$ where $e_{p_i}$ is the $p_i^{\mathrm{th}}$ columns of the identity matrix $I_m\in {\mathbb{R}}^{m\times m}$ for $i=1,\cdots ,\ell$, are selected inductively from the basis $\{u_1,\cdots ,u_{\ell }\}$ by the DEIM algorithm; see [25]. By using the DEIM algorithm, it is guaranteed that $P^T{U}_{\ell }$ is always nonsingular.

Note that the DEIM approximation is uniquely determined by the projection matrix $U_{\ell }$, and $\widehat{f}$ is an interpolation approximation for the original function f, since $\widehat{f}$ is exact at the interpolation indices, i.e.,  it holds that $P^T\widehat{f}\left(V_k\tilde{x}\left(t\right)\right)=P^{T}f\left(V_k\tilde{x}\left(t\right)\right)$ for all $t\in [0,T]$.

DEIM solves the computational complexity problem that occurs since the nonlinear term has to be evaluated by replacing the orthogonal projection of POD with an interpolatory projection. As a result, the nonlinear term has to be assessed only at a few selected component functions, which greatly reduces the complexity. Note that the choice of the snapshots is a crucial factor in the construction of the POD basis, and the choice of the projection basis greatly affects the accuracy of the approximation of the original solution space. The snapshots are numerically sampled values of the trajectory $x(.)$ at particular time steps for a particular initial condition and at particular parameter values. The DEIM state approximation error for large systems of nonlinear ODEs was analyzed in [28], and a bound on the global state space approximation error was provided.

3. Application of the POD-DEIM Approach to Kinetic Model Examples

In this section, we apply the POD-DEIM model reduction method introduced in the previous section to kinetic models for different examples of biological systems. In particular, we consider the simple metabolic-genetic network example of [26], a large-scale kinetic model of the yeast metabolic network taken from [30], and a large-scale kinetic model of the Escherichia coli metabolic network introduced in [31]. All results presented in the following sections were computed with MATLAB (R2017a) on a GNU Linux system (openSUSE Leap 15.0).

3.1. Kinetic Model of the Metabolic-Genetic Network

In this example, we consider the model of the simple metabolic-genetic network studied in [26]; see Figure 1. The network contains 20 reactions and 19 dynamic states: 7 external metabolites $(C_1,C_2,F_{ext},O_{ext},D_{ext},E_{ext},H_{ext})$, 11 internal metabolites $(A,B,C,D,E,F,G,H,ATP,NADH,O_2)$, and biomass. This sample metabolic network models the dynamic growth of a cell while seven of the reactions are regulated by four temporary regulatory constraints, and it can be used to illustrate several insightful scenarios (e.g., preferential carbon source uptake/catabolite repression, anaerobic growth, amino acid biosynthesis pathway repression, or transcriptional regulation to maintain the concentration levels of important metabolites; see [26]). For this metabolic network example, we introduced a kinetic model describing the dynamics of external metabolites, internal metabolites, and regulatory rules for the network; see the Appendix A.

In particular, the change of the external metabolite concentrations in the cell medium generates different scenarios. Here, we will discuss the two scenarios called the diauxic-switch and aerobic/anaerobic-diauxie scenarios, which were also studied in [26]. First, we apply MOR methods to the kinetic model of the metabolic genetic network (A2) in the diauxic-switch scenario.

Application of POD-DEIM to the Diauxic-Switch Scenario

In the case of the diauxic-switch scenario, there are two sources of carbon that are introduced to the cell. During the first phase, the cells prefer to metabolize the sugar on which they can grow faster (e.g., $C_1$). When the first sugar has been exhausted, the cells switch to the second source of carbon (e.g., $C_2$); see [32,33].

We use the MATLAB function ode23s with tolerances $RTOL=ATOL={10}^{-6}$ to compute a numerical solution of the ODE system (A2) with parameters as given in

Table A2. The set of all parameters of the model that are estimated of two datasets of the diauxic switch and aerobic/anaerobic-diauxie scenarios using the D2D toolbox.

Constant Rates	Value	Unit
$K_{mc1}$	$0.38$	mM	estimated
$K_{mc2}$	$0.38$	mM	estimated
$K_{mf}$	$6.2$	mM	estimated
$K_{m{o}_2}$	$5.6·{10}^{-5}$	mM	estimated
$K_{md}$	$1.1·{10}^{-4}$	mmol/gDW	estimated
$K_{me}$	${10}^{-5}$	mmol/gDW	estimated
$K_{mh}$	41	mM	estimated
${\tilde{k}}_1$	980	mmol3/gDW3 · h−1	estimated
${\tilde{k}}_2$	1000	mmol2/gDW2 · h−1	estimated
${\tilde{k}}_3$	300	mmol/gDW · h−1	estimated
${\tilde{k}}_4$	140	mmol3/gDW3 · h−1	estimated
${\tilde{k}}_5$	23	mmol/gDW · h−1	estimated
${\tilde{k}}_6$	25	mmol/gDW · h−1	estimated
${\tilde{k}}_7$	$1.6$	mmol/gDW· h−1	estimated
${\tilde{k}}_8$	1000	mmol/gDW· h−1	estimated
${\tilde{k}}_9$	13	mmol/gDW· h−1	estimated
${\tilde{k}}_{10}$	$2.9$	mmol2/gDW2· h−1	estimated
${\tilde{k}}_{11}$	150	mmol3/gDW3 · h−1	estimated
${\tilde{k}}_{12}$	$7.9·{10}^{-5}$	mmol/gDW· h−1	estimated
${\tilde{k}}_{13}$	170	mmol2/gDW2 · h−1	estimated
$\zeta$	${10}^{-5}$	mM	estimated
$\gamma$	$0.024$	mmol/gDW	estimated
$\beta$	$0.11$	mM	estimated
$\alpha$	290	mmol/gDW	estimated
$\omega$	1	gDW/mmol	assumed

in the time interval [0, 5] hr using a non-equidistant output time-grid. The initial concentrations of internal metabolites are assumed to be $(A,B,C,D,E,F,G,H,ATP,NADH,O_2)=(0,1,0,0,0,0.2,0,0.03,6,5,0)$. The initial concentrations of external metabolites are taken from [26] for the different scenarios. This numerical solution yields the snapshot matrix $\mathbb{X}$ and the nonlinear snapshot matrix $\mathbb{F}$. We used the MATLAB function svd to calculate the singular value decomposition of $\mathbb{X}$ and $\mathbb{F}$. The singular values are depicted in Figure 2.

For $\mathbb{X}$, we could observe gradually decaying singular values with a strong decay for the smallest ones, indicating that neglecting these values would not result in any considerable loss of information in the reduced order model. A similar behavior could be observed for $\mathbb{F}$. The original model had a dimension of $m=19$. We computed reduced order models of different dimensions, once using the POD approach and once using the POD-DEIM approach. In the first scenario, we used $k=\ell =16$ for both approaches. The simulation results of the reduced order system in comparison with the simulation results of the original system are depicted in Figure 3 for some selected components of the state vector. We could observe that the main features of the dynamical behavior of the original model states were preserved in both reduced order models.

In a second scenario, we reduced the dimension to $k=15$, $\ell =16$. The results are depicted in Figure 4.

In a third scenario, we reduced the dimension to $k=14$, $\ell =16$. The results are depicted in Figure 5.

We can observe that some curves still matched well with the original model, while the solutions moved off, after $t=4$ hr in the second scenario and after $t=3$ hr in the third scenario. The behavior of $C_1$, $O_{ext}$, and G still fit well for all reduced models.

Since biomass expresses the growth of the cell, it is an important component of the network that should also be preserved. We could observe that the reduced model preserved the biomass behavior quite well, such that we obtained approximately the same growth.

In the fourth scenario, we used $k=14$ and $\ell =13$. The results are depicted in Figure 6.

We could observe that the trajectories of $C_1$ and $O_{ext}$ still fit well, but deviations occurred for the other components. In particular, differences between the POD and the POD-DEIM models could be observed that suggested that information captured by the truncated SVDs was not sufficient to predict the original behavior. We also compared the time costs for the simulations; see

Table 1. Comparison of the computing times for the four different scenarios.

	Original Model	POD ROM	POD-DEIM ROM
Scenario 1 ($k=16$, $\ell =16$)	0.3 s	1.4 s	0.7 s
Scenario 2 ($k=15$, $\ell =16$)	0.3 s	1.4 s	0.6 s
Scenario 3 ($k=14$, $\ell =16$)	0.3 s	1.9 s	0.9 s
Scenario 4 ($k=14$, $\ell =13$)	0.3 s	1.9 s	0.9 s

. For evaluating the computing times, we used the MATLAB function timeit.

We could see that the calculation of the reduced order models required some additional computational effort, in particular the projections with $V_k^T$ and $V_k$. If the reduced order model captured the features of the original model well, the computational effort for POD-DEIM model was about half of the effort for the POD model. This example was at a small scale, and the payoff in terms of computational time is shown in the later examples.

3.2. Kinetic Model of the Yeast Metabolic Network

In [30], a workflow for converting metabolic reconstructions into large-scale kinetic models of yeast metabolism was developed. Its purpose was to take available datasets, perform a thorough analysis of the parameter constraints, and then, produce the kinetic model using large data integration. In this example, the generated kinetic model contained 281 metabolites. Here, we applied the order reduction method to the large-scale yeast model as described in [30]. The ODE model equations were taken from the BioModels database (http://identifiers.org/biomodels.db/BIOMD0000000496). Here, we used only some representative metabolites in order to compare the behavior of the original and the reduced order models. The snapshot matrix $\mathbb{X}$ was obtained from a simulation of the ODE model over the time interval [0, 3000] s using the MATLAB solver ode23tb with the default setting and initial values taken from the database model. The behavior of the singular values of the snapshot matrix is depicted in Figure 7. We could observe a fast decay in the singular values with a large number of values of magnitude ${10}^{-12}$ indicating that neglecting these values would not result in any considerable loss of information in the reduced order model.

The original ODE system was of dimension $m=281$, and we reduced it to $k=45$ (Scenario 1) and $k=40$ (Scenario 2). The results of the simulations are given in Figure 8 and Figure 9. A comparison of the computing times is given in

Table 2. Comparison of the computing times for the two different scenarios.

	Original Model	POD ROM
Scenario 1 ($k=45$)	0.28 s	0.20 s
Scenario 2 ($k=40$)	0.28 s	0.46 s

.

We could see that in Scenario 1, the metabolites showed the same behavior in both models. We selected here the same metabolites as were presented in the Supplementary Information of [30]. In Scenario 2, the behavior of the metabolites was still well preserved, although we could see that some of the curves started to oscillate. The beginning oscillations suggested that the reduced order system was unstable, while the original system was stable. The problem of the instability of the reduced order model was addressed, e.g., in [34] or [35] and could be circumvented by taking additional measures. These oscillations increased if we further decreased the dimension of the reduced order model until the numerical simulation became unstable. The time cost of the simulation could be lowered by a factor of by approximately 3/4 in Scenario 1, whereas the oscillations led to a higher computational effort in Scenario 2.

For this example, we only applied the POD approach, since we could not gain any improvement in the computational efforts using the DEIM approach. The reason here was that the selection of the equation required in the DEIM method could not be achieved for the model equations provided by the BioModels database.

3.3. Kinetic Model of the E. coli Metabolic Network

A method for the generation of genome-scale kinetic models of the E. coli organism from reconstruction data was proposed in [31]. Building a kinetic model requires kinetic parameters, fluxes, and rate laws. In [31], a small kinetic model was used with the presence of experimental data, which was then extended using typical estimates in cases where experimental data were not available.

We applied the POD method to the generated kinetic model of E. coli as presented in [31] containing $m=402$ state variables, and we compared the original and reduced model for some of the metabolites’ behavior. The ODE model equations were taken from the BioModels database (http://identifiers.org/biomodels.db/BIOMD0000000469).

The snapshot matrix $\mathbb{X}$ was obtained from a simulation of the ODE model over the time interval [0, 300] s using the MATLAB solver ode23tb setting $ATOL={10}^{-3}$ and the initial values taken from the database model. The behavior of the singular values of the snapshot matrix is depicted in Figure 10.

Again, we could observe a fast decay in the singular values with a large number of values of magnitude ${10}^{-13}$ indicating that neglecting these values would not result in any considerable loss of information in the reduced order model. The original ODE system was of dimension $m=402$, and we reduced it to $k=45$. The results of the simulations of some chosen metabolites are given in Figure 11 and Figure 12.

Here, we selected random metabolites for the comparison. We could observe that the metabolites showed the same behavior in both models. The time cost of the simulation in the reduced model could be lower by a factor of by approximately 1/7 compared to the original model, see

Table 3. Computing times for the POD method.

	Original Model	POD ROM
Scenario ($k=45$)	0.10 s	0.014 s

.

From the previous discussions, we could conclude that the additional time cost required for the computation of the projection in the reduced order model in the POD approach was negligible for large-scale systems and was greatly exceeded by a gain in efficiency and computational speed. Naturally, the POD method was more effective and of more use in large-scale models.

4. POD for Kinetic Models With Different Initial Conditions

In this section, we study the POD method for a dynamical system with different initial conditions. In a kinetic system model, different initial conditions can be used to describe different scenarios or different modes of a biological system. We followed the same steps of the POD approach as in Section 2. At first, we computed the snapshot matrix $\mathbb{X}=[x\left(t_1\right),x\left(t_2\right),\cdots ,x\left(t_{n_s}\right)]\in {\mathbb{R}}^{m\times n_s}$ by solving the system of ODEs (1) with the initial condition $x_0$. Then, we computed another snapshot matrix $\mathbb{Y}=[x\left(t_1\right),x\left(t_2\right),\cdots ,x\left(t_{n_s}\right)]\in {\mathbb{R}}^{m\times n_s}$ with a different initial condition $y_0$. We combined the two snapshot matrices $\mathbb{X},\mathbb{Y}$ in a snapshot matrix $\mathbb{Z}$ given in the following form:

$$\mathbb{Z}=\left[\mathbb{X}\mathbb{Y}\right]\in {\mathbb{R}}^{m\times n_{\widehat{s}}},n_{\widehat{s}}=2n_s.$$

Then, by applying the singular value decomposition, we obtained:

$$\mathbb{Z}=U\left[\begin{array}{cc}{\mathsf{\Sigma }}_r& 0\\ 0& 0\end{array}\right]V^T,$$

where $U\in {\mathbb{R}}^{n_m\times n_m}$ and $V^T\in {\mathbb{R}}^{n_{\widehat{s}}\times n_{\widehat{s}}}$ are unitary matrices with orthonormal columns. The term ${\mathsf{\Sigma }}_r=diag({\sigma }_1,{\sigma }_2,\dots ,{\sigma }_r)\in {\mathbb{R}}^{r\times r}$ is a matrix with real, non-negative entries on the diagonal and zeros off the diagonal, and $r=rank\left(\mathbb{Z}\right)$. By truncating the least dominant singular vectors and projecting the original system onto the lower-dimensional subspace, we obtain a reduced order model.

Application to the Kinetic Model of a Metabolic-Genetic Network

In the following, we apply the idea described above to the kinetic model of the metabolic-genetic network from [26] with model equations given in (A2) for different scenarios. The scenarios were the diauxic switch scenario and the aerobic/anaerobic-diauxie scenario, and both scenarios could be predicted using the kinetic model by prescribing specific initial values.

The model in the diauxic-switch scenario was already introduced in Section 3.1. We used the MATLAB function ode23s with tolerances $RTOL=ATOL={10}^{-6}$ and initial conditions

$$x_0=[10,10,0,50,0,0,0,0.003,0,1,0,0,0,0.2,0,0.03,6,5,0]$$

to compute a numerical solution of the ODE system (A2) with parameters in the time interval $[0,5]$ h using a non-equidistant output time-grid. This numerical solution yielded the snapshot matrix $\mathbb{X}$.

In the aerobic/anaerobic-diauxic scenario of transcriptional regulatory modeling, there was only one source of carbon (i.e., $C_2>0$ and $C_1=0$) and oxygen supplied to the culture. After some time, oxygen was removed [26]. We used the MATLAB function ode23s with tolerances $RTOL=ATOL={10}^{-6}$ and initial conditions

$$y_0=[0,10,0,2,0,0,0,0.0008,0,1,0,0,0,0.2,0,0.03,6,5,0]$$

to compute a numerical solution of the ODE system (A2) with parameters in the time interval [0, 5] hr using a non-equidistant output time-grid. This numerical solution yielded the snapshot matrix $\mathbb{Y}$.

We used the MATLAB function svd to calculate the singular value decompositions of $\mathbb{Z}=\left[\mathbb{X}\mathbb{Y}\right]$. The singular values are depicted in Figure 13.

The original model had a dimension of $m=19$. We computed a reduced order model of dimension $k=17$ using the POD approach. The results of a simulation of the reduced order system in comparison with the simulation results of the original system are depicted in Figure 14 and Figure 15 for the two different scenarios, diauxic switch and aerobic/anaerobic-diauxie, respectively.

We could observe that the main features of the dynamical behavior of the original model states were preserved for both scenarios using the same reduced order model. This was an important result, as it showed that as long as the snapshot matrix provided the essential information, a reduced order model had to be computed only once and could then be used to predict the dynamical behavior of the system for very different scenarios. The magnitude by which the system could be reduced depended on the decay of the singular values. Singular values of magnitude ${10}^{-15}$ could be neglected, while singular values of order ${10}^{-4}$ still contained important information. For large-scale networks as in Section 3.2 and Section 3.3, the difference in the dimension of the reduced and the original model could be assumed to be much larger.

5. Conclusions

In this paper, we discussed the model reduction for kinetic models of biological networks using the POD and the POD-DEIM approach. Since kinetic models of chemical networks can be very large, the possibility to obtain a reduced order model that replicates the desired dynamical behavior is of vital importance. We applied the model reduction techniques to several examples. It has to be noted that the computation of a reduced order model requires some additional computational effort. However, the computation of the snapshots and the SVD usually has to be computed only once (often, this is called the offline phase), while the simulations of the reduced order models are run several times, e.g., within an optimization process (in the so-called online phase). Thus, the effort for setting up a reduced order model will pay off if we can reduce a large-scale model to a much smaller dimension and run simulations of the reduced order model for a long time. Furthermore, the efficiency of the model order reduction strongly depends on the decay of the singular values of the snapshot matrix. How far the model can be reduced also depends on the application and on what components of the solution one is interested. In addition, we showed that the dynamical behavior of a biological network can be predicted for very different scenarios (diauxic switch and aerobic/anaerobic-diauxie) using the same reduced order model.