Parallelization of Runge–Kutta Methods for Hardware Implementation

Abstract

Parallel numerical integration is a valuable tool used in many applications requiring high-performance numerical solvers, which is of great interest nowadays due to the increasing difficulty and complexity in differential problems. One of the possible approaches to increase the efficiency of ODE solvers is to parallelize recurrent numerical methods, making them more suitable for execution in hardware with natural parallelism, e.g., field-programmable gate arrays (FPGAs) or graphical processing units (GPUs). Some of the simplest and most popular ODE solvers are explicit Runge–Kutta methods. Despite the high implementability and overall simplicity of the Runge–Kutta schemes, recurrent algorithms remain weakly suitable for execution in parallel computers. In this paper, we propose an approach for parallelizing classical explicit Runge–Kutta methods to construct efficient ODE solvers with pipeline architecture. A novel technique to obtain parallel finite-difference models based on Runge–Kutta integration is described. Three test initial value problems are considered to evaluate the properties of the obtained solvers. It is shown that the truncation error of the parallelized Runge–Kutta method does not significantly change after its known recurrent version. A possible speed up in calculations is estimated using Amdahl’s law and is approximately 2.5–3-times. Block diagrams of fixed-point parallel ODE solvers suitable for hardware implementation on FPGA are given.

1. Introduction

Simulating continuous systems in discrete computers usually requires the numerical solution of ordinary differential equations (ODEs), which is usually performed using numerical integration methods [1,2,3,4]. Software implementing numerical integration methods is called an ODE solver. One of the critical characteristics of the ODE solver is its computational efficiency, usually determined as the dependence between the precision of the achieved solution and the time taken to obtain it [5]. Explicit Runge–Kutta (RK) methods are among the most popular solvers and belong to the broad class of single-step integration. They possess satisfactory numerical stability and high computational efficiency, being a reliable and straightforward tool for simulation software [6]. However, the hardware implementation of high-order RK methods is complicated due to the recurrent nature of their algorithms. This especially stands for hardware platforms operating with short and\or fixed-point data types where scaling procedures become complicated. Moreover, the abovementioned issue makes RK methods relatively poorly suitable for parallel computing and limits their application mainly to CPU-based software solutions.

Recently, so-called hardware-targeted methods gained great attention from scholars. Butusov et al. proposed a semi-implicit modification of extrapolation methods [7,8], which assumes method-based parallelism following the Aitken–Neville extrapolation. One of the most popular hardware platforms for implementing ODE solvers is a field-programmable gate array (FPGA), which possesses hardware-level calculation parallelism [9,10].

Many authors aimed to develop parallel versions of standard Runge–Kutta methods. Liu et al. proposed a variant of the parallel fourth-order explicit RK method [11]. Ding et al. proved the convergence of parallel Runge–Kutta methods and studied their application for solving delay differential equations (DDEs) [12]. Fei et al. developed a new class of parallel Runge–Kutta methods for solving differential-algebraic equations (DAEs) [13]. However, the abovementioned authors did not consider the hardware-targeted ODE solvers based on parallel RK methods and their applications.

Meanwhile, there are many practical applications of parallel ODE solvers based on RK methods known from the literature. The parallel implementation of the Runge–Kutta method was used, for example, in [14] for the numerical study of super-conducting processes in a system of Josephson junctions. Volokhova et al. applied parallel RK methods for computer simulation of the passage of a multicomponent gas-condensate mixture through a porous medium [15] and many others. Therefore, developing new techniques to parallelize RK methods is of particular interest in the field.

In this paper, we present an approach to parallelize explicit Runge–Kutta methods, following more general ideas by Tang et al. [16]. We explicitly show that the proposed solution is suitable for both software and hardware implementation. The rest of the paper is organized as follows. In Section 2, a brief description of the streaming problem is given and the pipeline processing principle is described. In Section 3, we describe the proposed technique using two representative examples. The truncation error of the parallelized RK method is compared with the original recurrent version of the solver and the possible calculations’ speedup is estimated. Finally, Section 4 provides some conclusions. In addition, some figures depicting the structure of designed double-precision and fixed-point ODE solvers are shown in the Appendix A.

2. Materials and Methods

In this paper, we propose a technique for modifying explicit Runge–Kutta solvers to be more suitable for parallel implementation. Let us allocate a class of streaming algorithms and some computational techniques first.

In the general case, the problem of streaming algorithms can be formulated as follows. Suppose there is an ordered set of vector data $D_i\langle d_1^i,d_2^i,\dots ,d_k^i\rangle ,\left(i=1, 2,\dots , N\right)$, all elements of which must be processed by an algorithm, visually represented by graph G(V,X) (Figure 1), where every vertex $v_j\in V$ is assigned an operation $O_j$ from an array of possible operations $O$. Arcs $\left(v_j,v_{j+1}\right)\in X$ of the mentioned graph are directed edges, which state that result, obtained from operation $O_j$ is simultaneously an input of an operation $O_{j+1}$.

The main idea of streaming algorithms is to transform input data vector $D_i$ $\left(i=1,2,\dots , N\right)$ into resulting output vector of data $R_i\langle r_1^i,r_2^i,\dots ,r_p^i\rangle ,\left(i=1, 2,\dots , N\right)$ according to the algorithm graph G(V,X).

Processors based on streaming algorithms with no integrated parallelism are being programmed to sequentially execute operations $O_j\left(i=1,2,\dots ,M\right)$ based on input vector $D_i$ $\left(i=1,2,\dots , N\right)$ (Figure 2).

Total processing time for N input vectors can be calculated using the following formula:

$$T_{comp}=N{\displaystyle \sum }_{i=1}^{M}f\left(O_i\right)\tau ,$$

(1)

where $M=\left|V\right|$ is the number of vertices on algorithm graph $G\left(V,X\right)$; $f\left(O_i\right)$ is the number of computer cycles required to perform operation $O_i$; $\tau$ is the duration of one cycle.

The main idea used to decrease computation time is to split the set of input vectors $D_i$ $\left(i=1,2,\dots , N\right)$ into N/C disjoint subsets, where C is the number of processors $P_i$ $\left(i=1,2,\dots ,C\right)$. Every processor $P_i$ $\left(i=1,2,\dots ,C\right)$ is configured to work with the set of operations $O_j\left(i=1,2,\dots ,m\right)$ and can be used to process one of the disjoint subsets independently. This allows for performing parallel computing on both software and hardware implementations of algorithms (Figure 3).

Thus, the total computation time required to obtain a solution can be calculated as follows:

$$T_{comp}=N/C{\displaystyle \sum }_{i=1}^{M}f\left(O_i\right)\tau ,$$

(2)

which is theoretically C-times less than the time required to obtain the same result without using parallel computing.

The computational efficiency can also be improved by using pipeline processing. The resulting structure can be described as an algorithm being divided into H consecutive operational stages. Thus, the amount of input and output data channels is decreased, but the whole process requires having a continuous flow of input data to the first operational stage, while it simultaneously does not have much effect on algorithms that have $M \gg N$.

If one wants to increase the computation speed of an algorithm and maintain the pipelining benefits, it is possible to improve the algorithm by adding parallelism to every operational stage (Figure 4).

Thus, the final computation time can be calculated as follows:

$$T_{comp}=\left(N+H-1\right)\Delta T,$$

(3)

and for the proposed example can be summarized as

$$\Delta T=\underset{i=1\dots H}{\mathit{max}}\left(T_{comp}^i\right)=\underset{i=1\dots H}{max}\left(\underset{j=1\dots M}{max}f\left(O_j^i\right)\tau \right)=\underset{i=1\dots M}{max}f\left(O_i\right)\tau ,$$

(4)

where $\underset{i=1\dots M}{max}f\left(O_i\right)$—is the most computationally demanding operation.

In case when N is significantly large and $M_i\approx \frac{M}{H}$, Equation (4) can be reorganized as:

$$T_{comp}=N\frac{{\sum }_{j=1}^{M}f\left(O_j\right)\tau }{\frac{HM}{H}}=\frac{H}{N}{\displaystyle \sum }_{j=1}^{M}f\left(O_j\right)\tau .$$

(5)

Thus, the total computation time can potentially be reduced by $\frac{M}{H}$ times, where M is the number of vertices of G(V,X) and H is the number of operational stages. The proposed scheme will be effective only in cases when all the vertices of the subgraph $G\left(V_i,X_i\right)$ are informationally independent and can be implemented simultaneously (parallel).

In the general case, the operational vertices of subgraphs $G\left(V_i,X_i\right)$ are information dependent, which means that some vertices $v_j\in V_i$ cannot be realized until the result of the vertex $v_{j-1}\in V_i$ realization is obtained. Therefore, the maximum rate of pipeline processing and, as a consequence, the $T_{comp}$ of processing the entire set of input data can be ensured if the structure of connections between the stages of the pipeline is fully adequate to the topology of connections of vertices in the algorithm graph $G\left(V,X\right)$.

This way of organizing pipelined computations is called structural. The maximum processing rate can be achieved if each operational vertex $v_i\left(i=1,2,\dots ,M\right)$ of the algorithm graph is assigned its own processor element $P_i\left(i=1,2,\dots ,M\right)$ and links between $P_i$ are provided according to the topology information arcs of the graph $G\left(V,X\right)$ (Figure 5).

If the vectors $D_i\left(i=1,2,\dots ,N\right)$ of the input set are sequentially fed to the input of such a computational structure, then their processing time will be $T_{comp}=\left(N+H_{max}-1\right)\Delta T$, where $H_{max}$ is the number of vertices on the critical path in the graph $G\left(V,X\right)$. The critical path is the path for which the value of $\sum _{j=1}^{M}f\left(O_j\right)\tau$ is maximum; $O_j\left(j=1,2,\dots ,k\right)$ are the operations belonging to the vertices of the critical path.

$\Delta T=\underset{j=1\dots H}{max}f\left(O_j\right)\tau$ is the execution time of the longest operation belonging to the vertices of the critical path. With a large value of N and considering that $\Delta T=\underset{i=1,2,\dots ,H_m}{\max }f\left(O_j\right)\approx N\frac{{{\displaystyle \sum }}_{j=1}^{M}f\left(O_j^i\right)}{M},$ one can obtain:

$$T_{comp}=\frac{N}{M}{\displaystyle \sum }_{j=1}^{M}f\left(O_j\right)\tau .$$

(6)

Analyzing Equation (6), one can conclude that using the structural method of organization, it is possible to minimize the computation time in comparison with the other methods of organizing computing systems considered above (see (1), (2) and (5)).

The scheme shown in Figure 5 represents the proposed computational structure, which combines both parallel and pipelined methods of processing since the input data are processed simultaneously along different pipelined processor chains. The proposed structure is called a multi-pipeline computational structure and is considered the most efficient for solving stream problems [17].

3. Results

In this study, we consider a parallel modification of RK4—explicit Runge–Kutta method of accuracy order 4 [18,19], which is a standard ODE solver in most popular simulation packages, e.g., MATLAB, Wolfram Mathematica, and NI LabVIEW software.

3.1. Test Problem 1: A Simple Linear System

Let an initial value problem (IVP) be specified as follows:

$$\frac{dy}{dt}=f\left(y,t\right),y\left(0\right)=y_0.$$

(7)

Pick a step size $h>0$ and use recurrent formulas of the RK4 method as follows:

$$k_1=f\left(y_i,t_i\right);$$

(8)

$$k_2=f\left(y_i+\frac{h}{2}{k}_1,t_i+\frac{h}{2}\right);$$

(9)

$$k_3=f\left(y_i+\frac{h}{2}{k}_2,t_i+\frac{h}{2}\right);$$

(10)

$$k_4=f\left(y_i+h{k}_3,t_i+h\right);$$

(11)

$$y_{i+1}=y_i+\frac{h}{6}\left(k_1+2k_2+k_3+k_4\right).$$

(12)

One can see that every formula from Equations (8)–(12) should be calculated sequentially, which excludes the possibility to organize parallel computing. Let us transform Equation (12) by substituting $k_1$ into the right side of (9), then substituting the calculated $k_2$ into (10), etc. To simplify the case, let us consider a simple ODE, which can be written as follows:

$$f\left(y,t\right)=ay+bx\left(t\right),$$

(13)

where $x\left(t\right)=t$, $y_0=1$, a = 1 and b = 1. For this expression, Equation (12) can be rewritten taking (7) as (13) and expressing a common denominator:

$$\begin{gathered} y_{i+1}=\frac{a^4{h}^4}{24}{y}_i+\frac{a^{3}b{h}^4}{24}{x}_i+\frac{a^3{h}^3}{6}{y}_i+\frac{a^{2}b{h}^3}{12}\left(x_i+\frac{h}{2}\right)+\frac{a^{2}b{h}^3}{12}{x}_i+\frac{a^2{h}^2}{2}{y}_i+ \\ \frac{ab{h}^2}{3}\left(x_i+\frac{h}{2}\right)+\frac{ab{h}^2}{6}{x}_i+ah{y}_i+\frac{bh}{6}\left(x_i+h\right)+\frac{2bh}{3}\left(x_i+\frac{h}{2}\right)+bh{x}_i+y_i. \end{gathered}$$

(14)

Now, it is possible to express another common denominator to slightly decrease the number of mathematical operations and obtain the correct form, which can then be used for parallel hardware implementation:

$$\begin{gathered} y_{i+1}=y_i\left(\frac{a^4{h}^4}{24}+\frac{a^3{h}^3}{6}+\frac{a^2{h}^2}{2}+ah\right)+x_i\left(\frac{a^{3}b{h}^4}{24}+\frac{a^{2}b{h}^3}{12}+\frac{ab{h}^2}{6}+bh\right)+ \\ \left(x_i+\frac{h}{2}\right)\left(\frac{a^{2}b{h}^3}{12}+\frac{ab{h}^2}{3}+\frac{2bh}{3}\right)+\frac{bh}{6}\left(x_i+h\right)+y_i. \end{gathered}$$

(15)

One can see that Equation (15) contains constants that can be excluded from parentheses and calculated preliminarily. This not only decreases the number of mathematical operations on every step of integration but allows one to use the proposed algorithm within the multi-pipeline computational structure (Figure A1).

ODE (13) can be solved analytically:

$$y\left(t\right)=\frac{b\left(-at+e^{at}-1\right)}{a^2}+e^{at}.$$

This analytical solution will be used to compare the original Runge–Kutta 4 method implementation with the proposed scheme. One can see from Figure 6 that the proposed parallel implementation maintains the truncation error of the conventional method.

One can see that the proposed structure (Figure 5) is suitable for hardware implementation using programmable logic devices and allows for organizing the computational process following the general principles of parallelism. In our study, we developed the concept of a hardware parallel ODE solver using NI FPGA technology. However, to perform this, one needs to apply a scaling procedure and turn the algorithm to using fixed-point arithmetic.

Conversion of the developed ODE solver with double data type (see Appendix A, Figure A1) to the solver with the fixed-point data type (Figure A2) was carried out using the scaling technique, previously reported in other works [20,21]. Following this methodology, in order to set the size of the integer and fractional parts, it is first necessary to determine the minimum and maximum possible value of each variable for the state of the system. To achieve this, a preliminary simulation of the system with the ODE solver of the double data type was carried out. Since the solution is exponential, it was decided to introduce restrictions on the simulation time for the integer ODE solver. In the course of the preliminary simulation, it was found that the optimal simulation time for a data type with a 32-bit machine word is an interval from 0 to 10 s. At this time interval of the solution, 17 bits are required to store the integer part of the state variables and parameters. Thus, all state variables and constant solver coefficients were converted to the FXP data type, where 1 bit is allocated to store the sign of the number, 17 bits are allocated to store the integer part, and the remaining bits are allocated to the fractional part. In the case of the fixed-point data type with a word length of 32 bits, 14 bits are allocated to the fractional part, which ensures that the solution’s accuracy compared to the original solver of the double data type will equal approximately ${10}^{-4}$.

The proposed scheme (Figure A2) allows one to significantly increase computational efficiency, arranging six processors, five of which work parallel, in a cascade manner. However, it is impossible to directly compare the performance of a serial solver implemented on a PC and parallel solver implemented in a FPGA. Thus, we used Amdahl’s law [22] to theoretically assess a possible increase in computational efficiency:

$$S_p=\frac{1}{\alpha +\frac{1-\alpha }{p}},$$

(16)

where $\alpha$ is the percent of non-parallelizable operations and $p$ is the number of processors. In the considered case, we have six processors, which are performing mathematical operations, while five of them are parallel and one uses their output data vectors and cannot be parallelized. To simplify calculations, let us state that they are performing the same operation of vector multiplication. Thus, using Formula (16), one can state that the potential increase in efficiency for a single non-parallel operation out of six operations performed by six different processors is:

$$S_p=\frac{1}{0.16+\frac{1-0.16}{6}},$$

which gives 3.(33) for one step of integration. The obtained results show that the proposed technique allows one to significantly increase the performance of explicit Runge–Kutta solvers implemented in computers with non-von Neuman architecture. By comparing the performance of solvers for the IVP in the form of (13), one can make a general assumption on the potential performance increase. The explicit RK4 method in this case requires three non-parallelizable operations for every evaluation of the RHS function; thus, all four steps of the recurrent calculation plus the final step of $y_{n+1}$ point evaluation require $3+5+5+4+7=24$ subsequent mathematical operations. The proposed scheme (Figure A2) divides the process into five parallelizable blocks of operations, where the most time and resource consuming blocks #2 and #3 require five operations each; thus, according to the idea of multi-pipeline structure, the general time to calculate 14 operations will be the same as it takes for 5 operations. The total amount of subsequent operations in that case equals $5+4=9$. Considering that time required to perform a single operation of either addition or multiplication on a middle-end machine with 8 core CPU is around 1,3E-6 s, one can make a general assumption on the possible performance increase by calculating the total time for a one-step evaluation using both classical and parallel implementations. Thus, for an IVP (13), total time elapsed on one evaluation step is around 0.0000312 and 0.0000117 s for classical and proposed parallel implementations of RK accordingly, which leads to a conclusion that the performance increase in the case of a linear system (13) will be about 2.(66)-times in comparison with the original scheme.

The solution of the previous sample ODE is an exponentially growing function, but the proposed technique can be used for any kind of IVP. Thus, let us consider another example—the oscillator case.

3.2. Test Problem 2: System with Periodic Solution

One can apply the proposed parallelizing technique when solving classical ODE systems with periodical solution:

$$\{\begin{array}{c}\frac{dx}{dt}=y\\ \frac{dy}{dt}=-x\end{array}.$$

(17)

First, expand Runge–Kutta recurrent formulas for Equation (17):

$$x_{n+1}=x_n-\frac{h}{6}\left(h\left(x_n+\frac{h}{2}{y}_n\right)+\left(h\left(x_n+\frac{h}{2}\left(y_n-\frac{h}{2}{x}_n\right)\right)+\left(h{x}_n-6y_n\right)\right)\right)$$

$$y_{n+1}=y_n-\frac{h}{6}\left(h\left(y_n-\frac{h}{2}{x}_n\right)+\left(h\left(y_n-\frac{h}{2}\left(x_n+\frac{h}{2}y\right)\right)+\left(h{y}_n+6x_n\right)\right)\right)$$

$$x_{n+1}=x_n+h{y}_n-\frac{h^2{x}_n}{2}-\frac{h^3{y}_n}{6}+\frac{h^4{x}_n}{24}$$

$$y_{n+1}=y_n-h{x}_n-\frac{h^2{y}_n}{2}+\frac{h^3{x}_n}{6}+\frac{h^4{y}_n}{24},$$

and simplify it one more time to further reduce the amount of mathematical operations:

$$x_{n+1}=x_n+y_n\left(h-\frac{h^3}{6}\right)+x_n\left(\frac{h^4}{24}-\frac{h^2}{2}\right)$$

$$y_{n+1}=y_n+x_n\left(\frac{h^3}{6}-h\right)+y_n\left(\frac{h^4}{24}-\frac{h^2}{2}\right),$$

which leaves us with three coefficients:

$$a_1=\left(h-\frac{h^3}{6}\right),$$

$$a_2=\left(\frac{h^3}{6}-h\right),$$

$$a_3=\left(\frac{h^4}{24}-\frac{h^2}{2}\right).$$

$a_1$, $a_2$, and $a_3$ can be used in multi-pipeline computation, as shown in Figure A3, or be pre-calculated outside of the integration cycle to further increase computational efficiency, allowing one step of integration to be performed in eight sequential operations (von Neumann architecture, processors #3 and #4 in Figure A3) or two-times faster, having two processors computing results simultaneously (parallel hardware implementation in FPGA).

The comparison of accuracy for serial and parallel RK solvers obtained regarding analytical solution (18) is presented in Figure 7.

$$\begin{gathered} x\left(t\right)=\cos \left(t\right), \\ y\left(t\right)=-\sin \left(t\right). \end{gathered}$$

(18)

One can see from Figure 7 that for an oscillatory system of order 2, the proposed parallel ODE solver does not possess an increased truncation error. Moreover, the observed difference is mostly a phase shift and the amplitude component of error is negligible. Let us consider an IVP of order 3 as a third test problem.

3.3. Test Problem 3: Third-Order System

It is known that the computation time needed for simulating systems of ODEs strictly depends on the dimension of the system. Systems of higher order require higher computation time. This especially stands for a case when all of the mathematical operations are being performed sequentially, and the higher the order of the system, the more efficient the proposed parallelization technique becomes due to the increasing amount of calculations on every step of recurrent formulas we can parallelize. Let us consider an ODE system of order 3 to evaluate this.

$$\{\begin{array}{c}\frac{dx}{dt}=2x+y+z\\ \frac{dy}{dt}=x+2y+z\\ \frac{dz}{dt}=x+y+2z\end{array}$$

(19)

System (19) requires 27 sequential operations of multiplication and division and 33 operations of addition during the simulation cycle using classic recurrent Runge–Kutta formulas. Thus, having implemented parallel computation, it is possible to significantly decrease the computation time, having some of the operations either being performed outside of the simulation cycle or being computed using the proposed multi-pipeline computational structure. In this case, the parallelized Runge–Kutta formulas can be written as follows:

$$\begin{gathered} x_{n+1}=x_n+2h{x}_n+h{y}_n+h{z}_n+3h^2{x}_n+\frac{11h^3}{3}{x}_n+\frac{43}{12}{h}^4{x}_n+\frac{5}{2}{h}^2{y}_n \\ +\frac{7}{2}{h}^3{y}_n+\frac{85}{24}{h}^4{y}_n+\frac{5}{2}{h}^2{z}_n+\frac{7}{2}{h}^3{z}_n+\frac{85}{24}{h}^4{z}_n, \end{gathered}$$

$$\begin{gathered} y_{n+1}=y_n+h{x}_n+2h{y}_n+h{z}_n+3h^2{y}_n+\frac{11h^3}{3}{y}_n+\frac{43}{12}{h}^4{y}_n+\frac{5}{2}{h}^2{x}_n+\frac{7}{2}{h}^3{x}_n+ \\ \frac{85}{24}{h}^4{x}_n+\frac{5}{2}{h}^2{z}_n+\frac{7}{2}{h}^3{z}_n+\frac{85}{24}{h}^4{z}_n, \end{gathered}$$

$$\begin{gathered} z_{n+1}=z_n+h{x}_n+h{y}_n+2h{z}_n+3h^2{z}_n+\frac{11h^3}{3}{z}_n+\frac{43}{12}{h}^4{z}_n+\frac{5}{2}{h}^2{x}_n+\frac{7}{2}{h}^3{x}_n+ \\ \frac{85}{24}{h}^4{x}_n+\frac{5}{2}{h}^2{y}_n+\frac{7}{2}{h}^3{y}_n+\frac{85}{24}{h}^4{y}_n. \end{gathered}$$

Following the abovementioned ideas, one can simplify those formulas further, ending with 33 operations of multiplication and division and 15 operations of addition:

$$\begin{gathered} x_{n+1}=x_n+x_n\left(2h+3h^2+\frac{11}{3}{h}^3+\frac{43}{12}{h}^4\right)+y_n\left(\frac{5}{2}{h}^2+\frac{7}{2}{h}^3+\frac{85}{24}{h}^4+h\right)+ \\ {z}_n\left(\frac{5}{2}{h}^2+\frac{7}{2}{h}^3+\frac{85}{24}{h}^4+h\right), \end{gathered}$$

$$\begin{gathered} y_{n+1}=y_n+x_n\left(\frac{5}{2}{h}^2+\frac{7}{2}{h}^3+\frac{85}{24}{h}^4+h\right)+y_n\left(2h+3h^2+\frac{11}{3}{h}^3+\frac{43}{12}{h}^4\right) \\ +z_n\left(\frac{5}{2}{h}^2+\frac{7}{2}{h}^3+\frac{85}{24}{h}^4+h\right), \end{gathered}$$

$$\begin{gathered} z_{n+1}=z_n+x_n\left(\frac{5}{2}{h}^2+\frac{7}{2}{h}^3+\frac{85}{24}{h}^4+h\right)+y_n\left(\frac{5}{2}{h}^2+\frac{7}{2}{h}^3+\frac{85}{24}{h}^4+h\right)+ \\ {z}_n\left(2h+3h^2+\frac{11}{3}{h}^3+\frac{43}{12}{h}^4\right). \end{gathered}$$

Thus, we obtain the coefficients, which can be pre-calculated (in that case, one will require a total of nine multiplications and nine additions per step, which can then be split into three separate groups with the possibility of parallel computation for FPGA implementation) or taken into computation inside of the integration cycle following the principles of multi-pipelining (Figure A4):

$$a_1=\left(2h+3h^2+\frac{11}{3}{h}^3+\frac{43}{12}{h}^4\right),$$

$$a_2=\left(\frac{5}{2}{h}^2+\frac{7}{2}{h}^3+\frac{85}{24}{h}^4+h\right).$$

The analytical solution of the system (19) is known, which helps us to compare the results of the simulation with both sequential and parallel schemes:

$$\begin{gathered} x\left(t\right)=e^t\left(3e^{3t}-2\right), \\ y\left(t\right)=e^t+e^{4t}, \\ z\left(t\right)=e^t+e^{4t}. \end{gathered}$$

(20)

It was also of interest to analyze the accuracy order of the proposed scheme with initial values of $x_0=-1$,$y_0=2$,$z_0=2$ (Figure 8).

One can evaluate the potential speedup of the proposed solver in a parallel case using Amdahl’s law. It is expected to be not less than three times for five-thread pipelining architecture. For systems of higher order, the advantage can be even more sufficient. Evaluating the strict dependence between the order of the simulated system and the calculations’ speedup will be the topic of our further research.

4. Conclusions

In this paper, we proposed a novel technique to parallelize explicit Runge–Kutta methods for an efficient hardware implementation of ODE solvers within the pipelining architecture. We described the proposed approach as an algorithm, illustrating it by solving three representative examples of ODEs.

We used the estimation of the truncation error to prove that the reported parallel implementation method does not significantly affect the precision of the method. We demonstrated the possibility of reducing computational costs while fully maintaining the properties of the original method. A hardware ODE solver architecture for FPGA implementation was created using the fixed-point scaling technique. It is shown that the performance gain obtained by parallelizing explicit RK solvers will increase with the order of simulated systems. It should be noted that the proposed solver architecture significantly reduces the complexity of converting Runge–Kutta algorithms to fixed-point hardware.

The obtained results can be applied to the development of high-speed hardware simulation systems. It is of interest to investigate the possibility of creating a high-precision control system using the proposed parallelizing approach and developing adaptive step-size techniques for parallel Runge–Kutta methods. These topics will be the key directions of our further studies.