On a Variational Method for Stiff Differential Equations Arising from Chemistry Kinetics

Abstract

For the approximation of stiff systems of ODEs arising from chemistry kinetics, implicit integrators emerge as good candidates. This paper proposes a variational approach for this type of systems. In addition to introducing the technique, we present its most basic properties and test its numerical performance through some experiments. The main advantage with respect to other implicit methods is that our approach has a global convergence. The other approaches need to ensure convergence of the iterative scheme used to approximate the associated nonlinear equations that appear for the implicitness. Notice that these iterative methods, for these nonlinear equations, have bounded basins of attraction.

1. Introduction

We start with the following stiff differential problem:

$$\begin{array}{ccc}\hfill x^{\prime }\left(t\right)& =& f\left(x\right(t\left)\right),\hfill \\ \hfill x\left(0\right)& =& x_0.\hfill \end{array}$$

Due to its stiffness, it is convenient to use implicit Runge-Kutta methods for the time integration step. The error analysis and the existence and uniqueness of the solution obtained through the Runge-Kutta method were analyzed in [1,2]. However, a difficulty appeared related to the approximation of the nonlinear Runge-Kutta equations by iterative methods [2], since we need to find good initial guesses. We will point out these difficulties in the Motivation Section.

The variational approach that we will use is based on the error functional:

$$E\left(x\right)={\int }_0^T\frac{1}{2}{|x^{\prime }\left(t\right)-f\left(x\left(t\right)\right)|}^{2}dx,$$

(1)

that will be minimized among the absolutely-continuous path $x:(0,T)\to {\mathbf{R}}^N$ with $x\left(0\right)=x_0$. Note that if $E\left(x\right)$ is finite for one such path x, then automatically, $x^{\prime }$ is square integrable. The error functional in (1) is associated with the original problem in a natural way: it only has a local minimum that is the solution of the problem. This property will imply the global convergence of our approach based on optimality conditions and minimization schemes like (steepest) descent methods. We only need to approximate linear problems; therefore, we can use the well-developed theory of the convergence of Runge-Kutta methods for stiff linear problems.

We will focus our attention on kinetic chemical reactions modeled through stiff differential equations. We will provide mathematical expressions for the reaction rate for problems where this rate is widely varying [3,4]. The main goal of this article is to test the variational approach exposed before to solve this kind of problems.

2. Motivation

One of the simplest stiff problems is the following linear test differential equation,

$$x^{\prime }\left(t\right)=\lambda x\left(t\right),$$

(2)

where $\lambda$ is a complex number with Re $\lambda <<0$. This equation is used to test the A-stability and imposes restrictions on the step size of explicit methods. In order to point out the problems with the initial guesses in the use of implicit methods, we will consider a nonlinear modification of the problem in (2). We consider the following nonlinear variation,

$$\begin{array}{ccc}\hfill x^{\prime }\left(t\right)& =& \lambda x\left(t\right)+x^2\left(t\right),\hfill \\ \hfill x\left(0\right)& =& 1,\hfill \end{array}$$

at $t=1$. The explicit solution is given by the expression,

$$x\left(t\right)=\frac{\lambda e^{\lambda t}}{1+\lambda -e^{\lambda t}}.$$

In order to solve this problem, we can consider the classical trapezoid rule,

$$x_{n+1}=x_n+\frac{h}{2}(f\left(x_n\right)+f\left(x_{n+1}\right)).$$

The approximate solution of the problem in (2) obtained using this classical implementation is shown in Figure 1. To obtain these results, we have used $\lambda =-100$ (left) and $\lambda =-$10,000 (right). We can see that the method provides a bad approximation when the stiffness of the problem increases, as shown in Figure 1 on the right. This is due to the fact that the initial guess is outside of the basin of attraction of Newton’s method [1]. However, looking at Figure 2, we can see that the variational method converges in the two considered cases.

3. Some Theoretical Analysis

One initial assumption that we must make about $f:{\mathbf{R}}^N\to {\mathbf{R}}^N$ is its smoothness, so that $\nabla f:{\mathbf{R}}^N\to {\mathbf{R}}^{N\times N}$ is continuous. We will also assume global Lipschitzianity with Lipschitz constant $M>0$ ($|\nabla f|\le M$). Moreover, we will suppose that for every positive $C>0$ and small $ϵ>0$, there is a constant $D_{C,ϵ}>0$ so that,

$$|f(x+y)-f\left(x\right)-\nabla f\left(x\right)y|\le D_{C,ϵ}{\left|y\right|}^2,\left|x\right|\le C,\left|y\right|\le ϵ.$$

This regularity comes from the relation of our approach with optimality. On the other hand, these properties are verified for most of the important problems in chemistry kinetics.

The next theorem is a local existence result. Fortunately, if the function f verifies the hypotheses globally, applying this theorem to successively smaller intervals, we can derive a global strong convergence result.

Theorem 1. 

Our iterative procedure $x^{\left(j\right)}=x^{(j-1)}+y^{\left(j\right)}$, starting from any initial approximation $x^{\left(0\right)}$, where:

$${\left(y^{\left(j\right)}\right)}^{\prime }\left(t\right)-\nabla f\left(x^{(j-1)}\left(t\right)\right)y^{\left(j\right)}\left(t\right)=f\left(x^{(j-1)}\left(t\right)\right)-{\left(x^{(j-1)}\right)}^{\prime }\left(t\right)in(0,T),y^{\left(j\right)}\left(0\right)=0,$$

converges strongly in $L^{\infty }(0,T)$ and in $H^1(0,T)$ to the unique solution of the original problem.

All the mathematical models considered in this work will verify the theoretical hypotheses since they will have both gradients and Hessians bounded.

Sketch of the proof: 

 

We start with any initial approximation $x_{\left(0\right)}(\equiv x)$, for instance $x_{\left(0\right)}=x_0$ for all t, or $x_{\left(0\right)}\left(t\right)=x_0+tf\left(x_0\right)$. In order to reduce the error finding a descent direction, we compute the Gâteaux derivative of E in (1) at a given x in the direction y with $y\left(0\right)=0$. Namely,

$$E^{\prime }\left(x\right)y={\int }_0^T(x^{\prime }\left(t\right)-f\left(x\left(t\right)\right))·(y^{\prime }\left(t\right)-\nabla f\left(x\left(t\right)\right)y\left(t\right))dt.$$

In particular, we can propose to choose y such that,

$$y^{\prime }\left(t\right)-\nabla f\left(x\left(t\right)\right)y\left(t\right)=f\left(x\left(t\right)\right)-x^{\prime }\left(t\right)\mathrm{in}(0,T),y\left(0\right)=0.$$

This way it is clear that $E^{\prime }\left(x\right)y=-2E\left(x\right)$, and the (local) decrease of the error is of the size $E\left(x\right)$. Finding y requires solving the above linear problem. In some sense, this is like a Newton method. Through an induction process, we obtain the proof of the theorem. The complete proof of Theorem 1 can be found in [5]. □

4. Some Numerical Examples

In this section, we introduce some well-known chemical problems and apply our iterative scheme in order to obtain an approximation of the solution and to show the efficiency of the new method. The models are: the Robertson problem, the chemical Akzo Nobel problem, and the Hires problem. For the three models, our approach is convergent. Indeed, we have:

Corollary 1. 

The three models (Robertson, chemical Akzo Nobel, and Hires) verify the hypotheses of Theorem 1.

Proof. 

The three models considered in this section verify the theoretical hypotheses of the variational approach. Two of them are quadratic polynomials that have a constant Hessian, and the third is based on concentrations (all positive and bounded), in monomials and square roots. Therefore, all are sufficiently differentiable with bounded gradients and Hessians. □

In particular, the user can consider the variational method as a black box with theoretical guarantees. This is not the case of other approaches, like implicit Runge-Kutta, where only numerical evidence is presented. For a more general context, of the chemistry kinetics and its applications, we refer to [6].

The choice of initial conditions when integrating systems of ODEs is usually not trivial, especially when the equations are stiff, and hence, the solution is not easily predictable. Therefore, the introduction of a new method that, under reasonable hypotheses, allows for a global convergence is rather important in our opinion. The figures correspond to the approximation of our method using a stopping criterion when the error functional is smaller than ${10}^{-5}$, and the global convergence of our approach ensures the good approximation to the solution. In fact, our methods can be used to check the existence of a solution, since the method only diverges basically when the problem has no solution. In any case, the values and the plots are those expected for these problems.

4.1. Robertson Problem

The following chemical reaction process was introduced by Robertson in 1966 [7]:

$$\begin{array}{cc}\hfill A& \stackrel{k_1}{\to }B,\hfill \\ \hfill B+B& \stackrel{k_2}{\to }C+B,\hfill \\ \hfill B+C& \stackrel{k_3}{\to }A+C.\hfill \end{array}$$

The problem consists of three equations where $k_1$, $k_2$, and $k_3$ represent the rate constants and A, B, and C are the chemical species involved.

Under some conditions and applying the mass action law for the rate functions, the following mathematical model, which consists of a stiff system of three non-linear ordinary differential equations, is obtained,

$$\begin{array}{cc}\hfill y_1^{\prime }& =-k_1{y}_1+k_3{y}_2{y}_3,\hfill \\ \hfill y_2^{\prime }& =k_1{y}_1-k_2{y}_2^2-k_3{y}_2{y}_3,\hfill \\ \hfill y_3^{\prime }& =k_2{y}_2^2.\hfill \end{array}$$

(3)

In the previous system, $y_1\left(t\right),y_2\left(t\right)$, and $y_3\left(t\right)$ were the concentrations of the chemical species A, B, and C, respectively. The initial values at time $t=0$ can be represented by ${\left(y_{01},y_{02},y_{03}\right)}^T$. The values of the constants used in the test were $k_1=0.04$, $k_2={3.10}^7$, and $k_3={10}^4$, and the initial concentrations were $y_{01}=1$, $y_{02}=0$, and $y_{03}=0$. The time interval considered was $0\le t\le 40$. The stiffness of this problem is due to the large difference between the kinetic constants $k_i$, which produce a very quick initial transient, as can be seen in Figure 3. The convergence of the method has been theoretically proven in [5]. In Figure 3, it is shown that the method converges and approximates well the solution of the Robertson problem. The numerical value of the solution for the three components $y_1$, $y_2$, $y_3$ of the Robertson problem at $t=40$, using the variational method, can be seen in

Table 1. Results of the Robertson problem at $t=40$.

${\mathit{y}}_{\mathit{i}}$	Solution at $\mathit{t}=40$
$y_1$	0.72524595
$y_2$	0.00000933
$y_3$	0.28331313

.

4.2. Chemical Akzo Nobel Problem

This problem arose in Akzo Nobel Central Research (The Netherlands) [8]. The problem describes a chemical process, in which two species, FLBand ZLU, are mixed, while carbon dioxide is continuously added. The resulting species of importance is ZLA. The names of the different chemical species are fictitious due to the commercial competition in this field. The chemical reaction scheme is the following,

$$\begin{array}{cc}\hfill 2FLB+\frac{1}{2}C{O}_2& \stackrel{k_1}{\to }FLBT+H_{2}O\hfill \end{array}$$

$$\begin{array}{cc}\hfill ZLA+FLB& \underset{{\mathrm{k}}_2}{\overset{\frac{{\mathrm{k}}_2}{\mathrm{K}}}{\rightleftarrows }}FLBT+ZHU\hfill \\ \hfill FLB+2ZHU+C{O}_2& \stackrel{k_3}{\to }LB+nitrate\hfill \\ \hfill FLB.ZHU+\frac{1}{2}C{O}_2& \stackrel{k_4}{\to }ZLA+H_{2}O\hfill \\ \hfill FLB+ZHU& \rightleftarrows FLBT.ZHU\hfill \end{array}$$

$K_s$ represents the constant of equilibrium of the last equation and has the expression,

$$K_s=\frac{[FLB.ZHU]}{\left[FLB\right].\left[ZHU\right]}$$

Brackets represent concentrations. The velocities of the previous chemical equations are given respectively by:

$$\begin{array}{cc}\hfill r_1& =k_1·{\left[FLB\right]}^4·{\left[C{O}_2\right]}^{\frac{1}{2}},\hfill \\ \hfill r_2& =k_2·\left[FLBT\right]·\left[ZHU\right],\hfill \\ \hfill r_3& =\frac{k_2}{K}{k}_2·\left[FLB\right]·\left[ZLA\right],\hfill \\ \hfill r_4& =k_3·\left[FLB\right]·{\left[ZHU\right]}^2,\hfill \\ \hfill r_5& =k_4·{[FLB.ZHU]}^2·{\left[C{O}_2\right]}^{\frac{1}{2}},\hfill \end{array}$$

$F_{in}$ defines the inflow of oxygen per unit volume and is given by:

$$F_{in}=klA·\left(\frac{p\left(O_2\right)}{H}-\left[O_2\right]\right).$$

where $klA$ denotes the mass transfer coefficient, H is the Henry constant, and $p\left(O_2\right)$ represents the partial pressure of oxygen and is considered independent of $\left[O_2\right]$. $k_1$, $k_2$, $k_3$, $k_4$, K, $klA$, H, and $p\left(O_2\right)$ are constant parameters. The mathematical model consists of a stiff system of six non-linear ordinary differential equations.

$$\begin{array}{cc}\hfill y_1^{\prime }& =-2r_1+r_2-r_3-r_4,\hfill \\ \hfill y_2^{\prime }& =-\frac{1}{2}{r}_1-r_4-\frac{1}{2}{r}_5+F_{in},\hfill \\ \hfill y_3^{\prime }& =r_1-r_2+r_3,\hfill \\ \hfill y_4^{\prime }& =-r_2+r_3-2r_4,\hfill \\ \hfill y_5^{\prime }& =r_2-r_3+r_5,\hfill \\ \hfill y_6^{\prime }& =K_s·y_1·y_2·y_6.\hfill \end{array}$$

(4)

where the auxiliary variables $r_i$ and $F_{in}$ are:

$$\begin{array}{cc}\hfill r_1& =k_1·y_1^4·y_2^{\frac{1}{2}},\hfill \\ \hfill r_2& =k_2·y_3·y_4,\hfill \\ \hfill r_3& =\frac{k_2}{K}·y_1·y_5,\hfill \\ \hfill r_4& =k_3·y_1·y_4^2,\hfill \\ \hfill r_5& =k_4·y_6^2·y_2^{\frac{1}{2}},\hfill \\ \hfill F_{in}& =klA·\left(\frac{p\left(O_2\right)}{H}-y_2\right).\hfill \end{array}$$

The values of the parameters were $k_1=18.7$, $k_4=0.42$, $K_s=115.83$ $k_2=0.58$, $K=34.4$, $p\left(O_2\right)$ $k_3=0.009$, $klA=3.3$, $H=737$. Initially, 0.44 mol/liter of FLB were mixed with 0.007 mol/liter of ZHU. At the beginning, the carbon dioxide had a concentration of 0.00123 mol/liter. The time of simulation was 180 minutes. The relation between chemical and mathematical formulation was that the concentration of the different components in chemical formulation $\left[FLB\right]$, $\left[C{O}_2\right]$, $\left[FLBT\right]$, $\left[ZHU\right]$, $\left[ZLA\right]$, $[FLB.ZHU]$ was respectively $y_1$, …, $y_6$ in the mathematical formulation. The method was able to approximate the solution of this stiff problem. The convergence of the method can be seen in Figure 4.

Table 2. Results of the chemical Akzo Nobel problem at $t=180$.

${\mathit{y}}_{\mathit{i}}$	Solution at $\mathit{t}=180$
$y_1$	0.11493515
$y_2$	0.00118672
$y_3$	0.16486622
$y_4$	0.00331426
$y_5$	0.16571073
$y_6$	0.19396618

shows the numerical solution of the components of the chemical Akzo Nobel problem $y_1$, $y_2$, $y_3$, $y_4$, $y_5$, $y_6$, at $t=180$, when the variational method is used.

4.3. Hires Problem

Schäfer introduced this problem in 1975 [9]. It represents the high irradiance response (HIRES) of photomorphogenesis on the basis of the phytochrome. The chemical reactions have eight reactants. The problem can be formulated mathematically through a stiff system of eight non-linear differential equations.

$$\begin{array}{cc}\hfill y_1^{\prime }& =-1.7y_1+0.43y_2+8.32y_3+0.0007,\hfill \\ \hfill y_2^{\prime }& =1.7y_1-8.75y_2,\hfill \\ \hfill y_3^{\prime }& =-10.03y_3+0.43y_4+0.035y_5,\hfill \\ \hfill y_4^{\prime }& =8.32y_2+1.71y_3-1.12y_4,\hfill \\ \hfill y_5^{\prime }& =-1.745y_5+0.43y_6+0.43y_7,\hfill \\ \hfill y_6^{\prime }& =-280y_6{y}_8+0.69y_4+1.71y_5-0.43y_6+0.69y_7,\hfill \\ \hfill y_7^{\prime }& =280y_6{y}_8-1.81y_7,\hfill \\ \hfill y_8^{\prime }& =-280y_6{y}_8+1.81y_7.\hfill \end{array}$$

The initial values were given by the vector $y={\left(1,0,0,0,0,0,0,0.0057\right)}^T$. The end points of the time interval for each component have been arbitrarily chosen, as can be seen in the numerical results (Figure 5). For the components $y_1$, $y_2$, $y_3$$,y_4$, $y_7$, and $y_8$, we have chosen the interval $\left[0,5\right]$, and for the components $y_5$ and $y_6$, we have chosen the interval $\left[0,350\right]$. The problem has eight components, and it may be considered as a large number of components. In this case, the method also converges to the solution sought; see Figure 5. The numerical value of the solution for the eight components $y_1$, $y_2$, $y_3$, $y_4$, $y_7$$y_8$ at $t=5$ and $y_5$, $y_6$ at $t=350$ of the Hires problem, using the variational method, can be seen in

Table 3. Results of the Hires problem at $t=5$ for $y_1$, $y_2$, $y_3$, $y_4$, $y_7$, $y_8$ and at $t=350$ for $y_5$ and $y_6$.

${\mathit{y}}_{\mathit{i}}$	Solution at $\mathit{t}=5$
$y_1$	0.03209606
$y_2$	0.00657329
$y_3$	0.00464137
$y_4$	0.09110392
$y_7$	0.00572391
$y_8$	0.00005439
${\mathit{y}}_{\mathit{i}}$	Solution at $\mathit{t}=\mathbf{350}$
$y_5$	0.00053631
$y_6$	0.00115496

.

5. Conclusions

In stiff systems of ODEs, the restriction of the time step becomes severe, particularly when the simulation time is large. In this paper, the new variational method proposed has been successfully applied to several initial value problems arising from chemical reactions composed of large systems of stiff ODEs. Moreover, the variational method has the advantage of never getting stuck at local minima. It always converges to the solution regardless of the initialization. This is an important improvement over the methods that need an initialization that is close to the solution of the problem like the implicit Runge-Kutta methods.