Novel Techniques for a Verified Simulation of Fractional-Order Differential Equations

Verified simulation techniques have been investigated intensively by researchers who are dealing with ordinary and partial differential equations. Tasks that have been considered in this context are the solution to initial value problems and boundary value problems, parameter identification, as well as the solution of optimal control problems in cases in which bounded uncertainty in parameters and initial conditions are present. In contrast to system models with integer-order derivatives, fractional-order models have not yet gained the same attention if verified solution techniques are desired. In general, verified simulation techniques rely on interval methods, zonotopes, or Taylor model arithmetic and allow for computing guaranteed outer enclosures of the sets of solutions. As such, not only the influence of uncertain but bounded parameters can be accounted for in a guaranteed way. In addition, also round-off and (temporal) truncation errors that inevitably occur in numerical software implementations can be considered in a rigorous manner. This paper presents novel iterative and series-based solution approaches for the case of initial value problems to fractional-order system models, which will form the basic building block for implementing state estimation schemes in continuous-discrete settings, where the system dynamics is assumed as being continuous but measurements are only available at specific discrete sampling instants.


Introduction
As mentioned in the abstract of this paper, verified (often interval-based) simulation techniques are widely known for sets of ordinary differential equation when computing the solution to initial value problems [1]. Throughout this paper, we employ the widely used notion of verification from the community of interval analysis, where a verified method deals with solution procedures that by construction contain all possible results in a guaranteed manner. Validation instead means that a mathematical model for a real-life application is consistent with experimental observations [2].
Although the dynamics of many system models in engineering applications can be analyzed with the methods mentioned above, the verified analysis of fractional-order systems is an emerging direction of research that is not yet fully solved. Fractional-order differential equations are characterized by the fact that not only integer orders of time derivatives appear in the state equations but also corresponding non-integer values are take into consideration [28][29][30][31]. If practical applications in (control) engineering are concerned, the applications for such dynamic system representations cover topics related to • Battery modeling and state-of-charge estimation [32][33][34]; • Fractional-order modeling and control of robotic manipulators aiming at enhanced loop-shaping [35,36]; • Modeling the formation of multi-robot systems, where each agent has integer-order dynamics but the overall behavior may result in a fractional-order behavior [37]; • Advanced modeling of visco-elastic damping [38]; • The generalization of classical PID and integral sliding-mode controllers [39][40][41][42]; • Modeling phenomena that cannot be explained fully by classical stress-strain relations (Newton's vs. Hooke's law) or diffusion vs. wave propagation phenomena [43,44].
If set-valued simulation techniques for fractional-order system models are concerned, especially the Mittag-Leffler function approach for computing enclosures for systems with stable dynamics should be pointed out [45,46]. It is based on generalizing the exponential enclosure technique [47,48]. This technique was originally developed by A. Rauh and E. Auer for integer-order dynamics. It is now generalized toward the use in fractional system models, where the Mittag-Leffler function is an extension of the exponential function. Both the exponential and Mittag-Leffler functions naturally represent closedform solutions of linear either integer-order or fractional-order system models that are specified by linear state equations [29,[49][50][51][52].
In this paper, Section 2 gives an overview of preliminaries and selected state-of-the-art techniques for fractional-order differential equations. These techniques are extended in Section 3 toward novel interval-based routines. The specific properties of these novel approaches are discussed in Section 4 for selected linear and nonlinear case studies. Finally, conclusions and an outlook on future work are given in Section 5.

System Models under Consideration
In this paper, we consider the simulation of in general nonlinear, commensurate fractional-order autonomous differential equations where the right-hand side f(x(t)) is assumed to be given by a continuous function. Furthermore, assume that 0 < ν ≤ 1 holds. Then, in the sense of a fractional-order system model of Caputo type [28,29], initial conditions are specified at the time instant t = 0. Uncertainty in the initial states x(0) is described in terms of the interval vector for which x i (0) ≤ x i (0) holds for each vector component i ∈ {1, . . . , n}.

Grünwald-Letnikov Approximation of Fractional Derivatives
One of the most classical approximations for the solution of fractional-order differential equations is the use of the Grünwald-Letnikov definition of fractional derivatives and, on its basis, the representation of the solution in terms of the infinite series with the step size ∆T k = t k+1 − t k .
Here, the term ( ν i ) represents the Newton binomial coefficient where the gamma function Γ(ν) serves as a generalization of the factorial for the case of real-valued arguments according to The severe disadvantage of this series definition in practical applications is its low rate of convergence. Hence, a large number of summands need to be accounted for in the approximation of the sum in Equation (3) to get sufficiently close to the true solution of the system model (1).
Note, in a verified setting, the truncated part of the infinite series cannot be ignored. In principle, however, it could be overapproximated with the help of the binomial series, cf. [53,54], dating back to Isaac Newton (around 1665) according to or, more generally, which would then have to be split up into a finite sum and into a remainder, where the values of x can be set to fixed values or interval bounds (often the initial condition). Due to the fact that (3) converges too slowly in practice and that bounding the remainder term as sketched above is not trivial and especially prone to overestimation in an interval setting, this option will not be considered further in this paper.

Single Time Step Picard Iteration
Further, quite recently investigated solution techniques for fractional-order differential equations are given by a Picard iteration procedure. As summarized in the two following theorems, taken from the cited literature without explicitly stating the proof in this paper, either point-valued approximations of the solution (Theorem 1) or interval-valued bounding boxes enclosing all solutions over a finitely long time span (Theorem 2) can be distinguished.
The following section of this paper will make use of a combination of the ideas of both theorems in the sense that a novel formulation which is close to Theorem 2 will be turned into an interval-valued iteration that produces bounding boxes of states for multiple subsequent time slices.
Theorem 1 ([55,56] Integral formulation of Picard iterations for fractional-order differential equations). Let f(x(t)) be a continuous Lipschitzian function on a bounded state and time domain. The solution to time-invariant fractional-order differential equations at the point of time T > 0 can be computed iteratively according to the fixed-point iteration with the initialization x 0 := x(0) at the iteration step κ = 0.
The convergence properties of the iteration in Theorem 1 are analyzed in detail in [56].

Theorem 2 ([55]
Interval-based, single time step Picard iterations for fractional-order differential equations). The iteration in Theorem 1 generalizes to interval bounded initial conditions where convergence requires As mentioned above, the numerical evaluation of Theorem 2 provides time-invariant bounds [x] κ+1 containing all possible solutions x(t) over the time interval t ∈ [0 ; T]. This fact limits the practical applicability to quite small values of T, which becomes obvious if, for example, ν = 1 was considered. Then, (9) would be nothing else than an explicit Euler evaluation of the solution with its corresponding well-known restrictions to small admissible values for T. Remark 1. Throughout this paper, the evaluation of functions for interval arguments such as in (9) denotes an arbitrary interval extension of the corresponding expressions. To reduce pessimism, these interval extensions should result in tight interval boxes without significant overestimation. This can be obtained by means of the exploitation of higher-order enclosure techniques or by making use of certain monotonicity properties of the functions to be evaluated (cf. [57][58][59]).

Special Case: "Closed-Form" Solutions
Consider-as a special linear case-the scalar fractional-order differential equation and the parameter λ ∈ R. According to [50,60], the exact solution of this linear system model of Caputo type is given by the expression where E ν,β (ζ) is the two-parameter Mittag-Leffler function with the general argument ζ ∈ C, the gamma function Γ(νi + β) as defined in (5), the derivative order ν, and β = 1 according to (11). An accurate floating-point evaluation of the function (12) becomes possible by the MATLAB implementation by R. Garrappa [51,61]. For the sake of completeness, although the procedure cannot be turned directly into a verified implementation using interval analysis, it should be pointed out at this position that numerical approximations of the solution of fractional-order differential equations can be obtained effectively with a predictor-corrector method implemented in the MATLAB routine fde12 [62] published also by R. Garrappa.

Verified Simulation Techniques
In this section, three novel procedures are introduced for a guaranteed, interval-based state enclosure technique that is applicable to fractional-order differential equations. Note, in the limit case ν = 1, they turn into results that are well-known from the field of verified simulation of classical sets of ordinary differential equations.

Mittag-Leffler Type State Enclosures for Fractional-Order Differential Equations
As stated in Section 2.4, closed-form solutions of linear fractional-order differential equations can be stated in terms of Mittag-Leffler functions. Therefore, the first investigated solution technique generalizes this solution representation towards the nonlinear case by computing parameters λ in (11) that are no longer point-valued but defined in terms of intervals with a finitely large width. In such a way, the solution representation according to Definition 1 can be employed for a verified simulation of fractional-order models that may include nonlinearities and bounded uncertainty in parameters and initial conditions. This solution approach makes use of an interval extension of Mittag-Leffler functions for real-valued arguments that was first published in [63,64].
Definition 1 (Mittag-Leffler type state enclosure). The time-dependent Mittag-Leffler type enclosure function  (13) contains all reachable states x * (T) at the point of time t = T > 0 according to with certainty if the diagonal elements of [Λ] are set to the outcome of the converging iteration Proof. According to [45,46], Picard iteration procedures for fractional-order systems cannot only be specified in integral form as in Theorems 1 and 2 but also in the equivalent differential form Substituting the Mittag-Leffler type enclosure of Definition 1 into (16) yields the expression Overapproximating the Mittag-Leffler type state enclosure in the iteration step κ + 1 on the first line of (17) which is based on the assumption leads to where The relation (21) then holds naturally due to inclusion monotonicity of interval analysis [57][58][59].

Remark 2.
A typical initialization of [λ i ] 0 is given by the real parts of the eigenvalues of the Jacobian of f at the midpoint of [x e ](0), inflated to small intervals with non-zero diameter. The inflation -currently implemented as the interval hull over [λ i ] κ and [λ i ] κ+1 -of these bounds is continued up to the point, where the condition (19) is satisfied.

Remark 3.
To achieve tight interval bounds, it is recommended to transform the state equations according to [45,46] into a coordinate frame in which they are decoupled as far as possible. For a time-invariant change of coordinates, the matrix of the eigenvectors of the Jacobian of f at the midpoint of [x e ](0) can be used if all of them are real-valued, see also [47,48].

Corollary 1.
For quasi-linear state-space representations of fractional-order differential equations in the form which replace the general dynamics (1) for systems with an equilibrium at x = 0 and well-defined, finite entries in the state-dependent matrix A(x(t)) and considering the initial conditions (2), Theorem 3 simplifies to the iteration scheme Remark 4. Note that for ν = 1 the quotient of two Mittag-Leffler functions in (23) can usually not be simplified further. Overestimation in the interval evaluation due to multiple dependencies on common interval parameters can be reduced by exploiting the monotonicity properties for Mittag-Leffler functions that were analyzed in detail in [64].

Remark 5.
For the order ν = 1, the iteration formulas in Theorem 3 and Corollary 1 become identical to the exponential enclosure technique published in [47,48] due to E 1,1 (x) ≡ e x .

Multi Time Step Picard Iteration
When using Theorem 1 to compute approximations of the solution to the system model (1) with (2), the integration that is desired for the horizon t ∈ [0 ; T] can be performed in intermediate steps for shorter ranges t ≤ T according to This reformulation leads to the interval formulation of the Picard iteration on the temporal discretization mesh 0 = t 0 < t 1 < . . . < t n = T according to the following theorem.
Theorem 4 (Interval-valued multi time step Picard iteration). A guaranteed interval enclosure of the state vector x(t n ) is obtained by the iteration if the temporally piecewise constant defined state bounds satisfy the relation Proof. Using (24), Formula (8) is cast into the interval-valued Picard iteration Assuming temporally piecewise constant defined interval bounds f κ i+1 i as it was done, for example, in [65], the integral in (28) can be bounded by Solving the remaining integral expression in (29) in closed form completes the proof of Theorem 4.

Remark 6.
For the integer-order case with ν = 1, this formulation becomes identical to the fundamental iteration scheme of VALENCIA-IVP, when setting the approximate solution introduced in [11] to zero.

Truncated Temporal Series Expansion
Theorem 5 (Truncated series enclosure for fractional-order differential equations). A truncated temporal series of length L with interval coefficients according to with the interval enclosure of the corresponding time derivative where is a guaranteed enclosure of all reachable states at t > 0 if it satisfies the relation Proof. The proof of this theorem consists of two parts. First, it is verified that differentiating (30) yields the expression (31). According to [28], the fractional-order derivative of the time-dependent polynomial (t − t 0 ) i , i ∈ N, is given by Here, t 0 is the point of time with respect to which the derivative is defined. Setting t 0 = 0 equal to the initial point of time at which the initial conditions are defined for the fractional-order system models in this paper, correctness of (31) is proven.
The remainder of the proof is a direct consequence of applying the differential formulation (16) of the Picard iteration into which both (30) and (31) are substituted. This is equivalent to the relation (33) and thus completes the proof of Theorem 5.

Remark 7.
For general nonlinear (non-polynomial) state Equations (1), interval Newton methods [57,66,67] can be employed to determine the coefficients [x (i+1)·ν ] and [R ] under consideration of (2) as additional constraint. For that purpose, either a temporal series expansion on both sides of (33) in the powers t i·ν is determined with subsequently extracting equalities for terms of identical orders on the left-and right-hand sides, or by determining a set of nonlinear algebraic equations after differentiating both sides with respect to the unknown coefficients. In future work, it will be investigated whether this truncated temporal series expansion can be used efficiently to determine interval enclosures for Mittag-Leffler functions with complex arguments. Such evaluations would be necessary if Theorem 3 was to be extended toward oscillatory systems in analogy to the complex-valued solution procedure that was introduced for integer-order dynamics in [48].

Remark 9.
For the case ν = 1, Theorem 5 represents an L-th order temporal Taylor series approximation of the solution of the considered initial value problem.

Numerical Case Studies
In this section, numerical examples are discussed for selected linear and nonlinear benchmark problems. Although only scalar differential equations are considered in Sections 4.1-4.4, straightforward extensions to higher-order models are possible as shown in [46] for the Mittag-Leffler type enclosure technique. Moreover, a nonlinear higherdimensional system model is considered in Section 4.5.
To make the Mittag-Leffler enclosure technique (Theorem 3) and the series expansion approach (Theorem 5) applicable without any further restrictions, it is assumed that the fractional derivative order ν is temporally constant in all cases in which it is specified as an interval parameter. Theorem 4, however, would also be valid directly for temporally varying, but interval-bounded derivative orders due to the conservative overapproximation of the temporal integral according to Equations (28) and (29). Note that the implementation of Theorems 3 and 5 uses an equidistant interval splitting in all function evaluations to reduce the effect of the interval-related dependency effect (cf. [57]), where 5-10 subdivisions were (heuristically) chosen for each interval quantity. This is not necessary for the considered examples when using Theorem 4, where overestimation can be countered at least partially by choosing small integration step sizes.
All following simulation results are obtained by using the MATLAB toolbox INT-LAB [68] which provides the required functionalities for interval analysis including verified implementations of standard functions and directed rounding.

Linear System with Point-Valued Parameters
As the first benchmark scenario, consider the linear fractional system model with precisely known initial conditions and parameters. The application of Theorem 3 naturally leads to the point interval [λ] = [λ] κ = [λ] κ+1 ≡ −[2 ; 2] as the outcome of the iteration procedure if its simplification according to Corollary 1 is applied. Hence, the solution according to Table 1 is given by the evaluation of the interval-valued Mittag-Leffler function [64] with Table 1 further contains the guaranteed lower and upper bounds computed by Theorem 4 at the same sampling instants after the maximum number of κ = 100 iterations as well as for the evaluation of Theorem 5 with the fixed order L = 20.  In addition, the decrease of the interval width, diam{[x](t)} = x(t) − x(t), at t = 0.1 is depicted in Figure 1a in dependence of the iteration parameter κ, while Figure 1b represents the dependence of the solution width on the approximation order L at the same point of time. Here, the interval-valued Picard iteration was evaluated on an equidistant discretization mesh with t i+1 − t i = 10 −3 . A plateau of approximately constant solution diameters is reached already after κ = 10 iterations. Hence, further work can implement step size control strategies which reduce the width of the discretization mesh to further enhance the solution quality. It can be noticed that the series-based solution representation outperforms the Picard iteration procedure with relatively small orders L < 10 for short integration horizons, while the Picard iteration is more accurate if longer time spans are accounted for. Especially for t > 0.5, the most accurate results are obtained by a direct evaluation of the interval extension of the real-valued Mittag-Leffler function.

Theorem 3 Theorem 4 Theorem 5 t x(t) x(t) x(t) x(t) x(t) x(t)
However, the reasonable accuracy of both alternatives from Theorems 4 and 5 motivates a possible direction for future research: In order to make the Mittag-Leffler type enclosures applicable for systems with complex eigenvalues, the Theorems 4 and 5 could be applied as already indicated in Section 3.3 to determine guaranteed enclosures of complex-valued Mittag-Leffler functions, for which the monotonicity properties derived in [64] no longer hold.

Linear System with Interval Parameters
The general observations from the previous subsection also hold for the uncertain linear system model with However, Table 2 and Figure 2 show the effect that the series expansion approach is more accurate for shorter time horizons as compared with the Picard iteration in a more pronounced way. Due to the fact that both are computationally efficient and simple to implement, it is advantageous to intersect the results with each other (admissible due to the fact that both represent guaranteed solution enclosures) to obtain more accurate simulation routines in future work. Such intersections can also be performed with the outcome of Theorem 3, which is recommended for either not fully decoupled linear models with n > 1 or in the nonlinear case that is addressed in the following subsections.  (38) and ν = 0.5.  To show the insensitivity of the Mittag-Leffler type enclosure technique on interval parameters [ν] for the fractional derivative order, see Table 3. Note, due to multiple interval dependencies on [ν], the other two solution techniques would need to be extended by techniques for reduction of interval-related overestimation (such as interval bisectioning). This is out of the scope of this paper, because the Mittag-Leffler type procedure is still more accurate than the solutions of both other alternatives, even if they are purely evaluated for a point-valued parameter ν.

Polynomial System with Point-Valued Parameters
For the point-valued nonlinear system model the multi time step Picard iteration is by far the most accurate approach according to the comparison in Table 4 as well as Figures 3 and 4. Moreover, it can be noted that the Mittag-Leffler type enclosures start to worsen for longer time horizons. This was already observed in [46], where a procedure for subdividing the integration horizon in combination with a guaranteed quantification of the arising truncation errors was introduced. Especially this routine could benefit from a combination of Theorem 3 with Theorem 4, where the latter would provide the reference solutions introduced in ([46], Equation (28)). The temporal series expansion approach alone, however, turns out to be inefficient in this case and is only applicable for very short integration horizons due to the strong nonlinearity. Therefore, the series definition in its current form should only be employed if the system under consideration has a dominant linear behavior.

Polynomial System with Interval Parameters
As a final benchmark, consider the nonlinear system model with the uncertain initial condition and parameter The uncertainty in this model turns the Mittag-Leffler type enclosures into the most efficient approach for this setting. This holds also if the fractional derivative order becomes uncertain, see Figures 5 and 6 as well as Table 5.
However, this result also opens up a further idea for future investigations. In the preceding sections, only the option to enhance the Mittag-Leffler type enclosures by means of Theorem 4 was discussed. However, it could be possible that one obtaines solution enclosures after a first intersection of the Picard iteration scheme with the results of Theorem 3 that can then be used in a second step for two further purposes: • Exploit advantages of Theorem 4 in early parts of the simulation (close to t = 0) to further improve Theorem 3; • Use this combination of both to further implement observer approaches, where measured data are only available at some discrete points of time.

Nonlinear Higher-Dimensional System Model
As a final nonlinear application scenario, consider the fractional-order epidemiological SEIR model with a compartmental structure according to [69] which can be used to forecast the development of infectious diseases within a population. The state equations of a corresponding fractional-order model can be stated by Here, the state variables are specified as follows: • S(t) denotes the susceptible part of a population; • E(t) the exposed group of the population; • I(t) the infectious group; • N(t) the non-constant total population size.
An important feature of this system model is the consideration that many diseases have a significant incubation period during which individuals have already been infected but are not yet infectious themselves. Those individual belong to the compartment E(t) before transitioning to I(t).
For the following simulations, the fractional derivative order is set to ν = 0.95 with the natural birth rate b = 0.001555, the vertical transmission parameters p = 0.8 and q = 0.95, the horizontal transmission rate r = 0.05, the rate β = 0.05 with which an exposed individual becomes infectious, the infection-related death rate θ = 0.002, the recovery rate γ = 0.003 and the state-dependent natural death rate For further details about this model and the choice of parameters, the reader is referred to [69] and the references therein.
Due to the non-negligible nonlinearity of this system model and the partially oscillatory dynamics observed in [69], the following simulations are restricted to the novel Picard iteration approach according to Theorem 4. The simulation case study takes into account three different sets of initial conditions: (44) Scenario 2: Consideration of a ±5% uncertainty in the initial conditions for the compartments of exposed and infectious individuals in Figure 8 with Scenario 3: Consideration of a ±1% uncertainty in the initial conditions for the compartments of exposed and infectious individuals in Figure 9 together with ±10% uncertainty in the susceptible class and population size with In all simulations, an equidistant discretization mesh with t i+1 − t i = 10 −1 has been employed.
The first scenario in Figure 7 clearly indicates that uncertainty due to the temporal discretization mesh leads to a blowup of the widths of the state enclosure after approximately 1000 integration time steps (t = 100). The state variables that are most affected by this blowup are the exposed and infectious compartments E(t) and I(t), respectively. It should be pointed out that the state equations according to (42) are implemented in such a way that overestimation due to multiple interval dependencies is reduced as far as possible by factoring out common state variables. However, the wrapping effect is not countered directly by the current implementation, which directly operates on the state equations in the form (42). Therefore, future work should try to reduce this type of pessimism by using solution representations that are less prone to the wrapping effect (for example, ellipsoidal enclosures [70,71]). Alternatively, procedures for a preconditioning of the state equations can be considered which aim at computing tighter solution bounds in a transformed set of coordinates [1,4]. Note, numerical simulations have shown that a further reduction of the integration step sizes does not lead to any noticeable decrease in the widths of the computed solutions for the scenario shown in Figure 7.
(d) Population size N(t). According to Figure 8, an increase in the uncertainty of the initial compartment sizes of exposed and infectious individuals firstly propagates to uncertainty in the group of susceptible persons. This kind of behavior can also be observed in Figure 9, where interval uncertainty was accounted for in all four state variables. To avoid the evaluation of the system models with states that are not reasonable, negative simulation results are cut off. This is justified by the proof given in [69] that all reasonable state variables lie with certainty in the interior of the positive orthant of the state space. Especially the two cases shown in Figures 8 and 9 point out a further direction for future research. Using the approach of interval observes, cf. [72] and the following conclusions section tightening of the solution enclosures will become possible on the basis of measured data. This will allow for a more efficient forecast of the propagation of diseases together with the possibility to estimate system parameters and to identify characteristics such as the death rate (43) in an online manner.

Conclusions and Outlook on Future Work
In this paper, it has been shown that Picard iteration procedures and temporal series expansion techniques can be employed for the development of interval-based simulation routines for fractional-order differential equations with nonlinearities and uncertain parameters. As such, they complement the Mittag-Leffler type solution technique that was already developed in previous work.
Combinations of all three proposed options will open up novel possibilities for enhancing solution enclosures not only in pure open-loop settings but also in cases where an interval observer technique such as in [72] is extended towards scenarios in which measured data are not available in continuous-time form but only at specific sampling instants. Due to the fact that the combination of various simulation techniques, as proposed in this paper, inevitably increases the required computational effort, the following two ideas could provide interesting directions for future research. Automatic recommender systems such as VERICOMP [73] could be generalized to the fractional-order case in order to automatically propose the most suited verified solver with its most appropriate settings. In addition, the CADNA software [74] (Control of Accuracy and Debugging for Numerical Applications) could be employed to determine those maximum numbers of iterations after which no solution improvements become possible because all resulting error terms purely result from numerical round-off errors.
In addition, future work should also deal with the development of estimation scheme that rely on the simulation procedures developed in this paper in order to suitably reconstruct initialization functions for fractional-order systems. As indicated, for example in [72], this is yet an open problem which is of specific practical interest if state estimation and simulation routines are not initialized at a system's equilibrium state.
Author Contributions: The algorithm was designed and implemented by A.R. The paper was jointly written by A.R. and L.J. Both authors have read and agreed to the published version of the manuscript.