Spectral Representations of Iterated Stochastic Integrals and Their Application for Modeling Nonlinear Stochastic Dynamics

Abstract

Spectral representations of iterated Itô and Stratonovich stochastic integrals of arbitrary multiplicity, including integrals from Taylor–Itô and Taylor–Stratonovich expansions, are obtained by the spectral method. They are required for the implementation of numerical methods for solving Itô and Stratonovich stochastic differential equations with high orders of mean-square and strong convergence. The purpose of such numerical methods is the modeling of nonlinear stochastic dynamics in many fields. This paper contains necessary theoretical results, as well as the results of numerical experiments.

1. Introduction

Differential equations are used to describe dynamical systems, whose states change in continuous time. The classical theory of differential equations assumes that their solutions are sufficiently smooth deterministic functions. However, to take into account random actions or disturbances, it is necessary to consider differential equations with random functions (random processes). If these random functions are sufficiently regular, then the classical theory can be applied to study solutions. For irregular random functions—such as white noise—the classical theory is not applicable, and, in its development, the theory of stochastic differential equations appeared [1,2,3]. Stochastic differential equations are used to describe stochastic dynamical systems in many fields [4,5,6,7].

In terms of applications, the methods for finding solutions to specific types of equations are important, but this is a very difficult problem. As a rule, when discussing the analytical solutions of Itô and Stratonovich stochastic differential equations, linear equations (or non-linear equations that are reduced to linear ones by a change in variables) are considered. Most of the examples of analytical solutions for stochastic differential equations are constructed via the inverse problem, i.e., according to the analytical expression of some random process. It is through this that the corresponding stochastic differential equation is derived. Since analytical solutions of stochastic differential equations can only be found in some particular cases, there is a need for methods that allow one to approximate solutions.

Starting with the Euler–Maruyama method [8], numerical methods for solving stochastic differential equations were developed. Various versions of Runge–Kutta-type methods were used in [9,10,11,12]. Rosenbrock-type methods were proposed in [13], and they included a regularization procedure, which makes it possible to not impose conditions on a step size for stability. The reviews of various methods for solving stochastic differential equations were published in [13,14,15,16,17,18,19,20].

There are numerical methods based on the expansions of the solution of stochastic differential equations: Taylor–Itô and Taylor–Stratonovich expansions [17,21], as well as the unified Taylor–Itô and Taylor–Stratonovich expansions [20]. These expansions are analogs of the Taylor series expansion of the solution of the ordinary differential equation. They contain iterated Itô and Stratonovich stochastic integrals over Wiener processes. The difference schemes for corresponding numerical methods contain the first terms of Taylor–Itô or Taylor–Stratonovich expansions, and the maximum multiplicity of iterated stochastic integrals in these terms determines the order of convergence for a specific numerical method. Although these expansions contain iterated stochastic integrals with the simplest weights (or, after transformations, with monomial weights), these integrals are generally not expressed in an elementary way in terms of random variables, for which there exist effective simulation algorithms.

For the approximate simulations of iterated Itô and Stratonovich stochastic integrals, we can use numerical integration; however, it is more preferable to apply the orthogonal expansion. This approach is described in [22], where the author approximates the iterated stochastic integrals of a second multiplicity using trigonometric functions. The numerical method using this approximation is called the Milstein method [13,17,20]. A similar approximation for the iterated stochastic integrals of a third multiplicity appeared in [17]. The approximation for integrals of a second multiplicity was discussed in [23,24], and their distribution was studied in [25]. In general, without taking into account iterated stochastic integrals in a difference scheme, it is impossible to obtain a high order of mean-square or strong convergence of the numerical method [26].

Significant progress in the approximation of iterated Itô and Stratonovich stochastic integrals of arbitrary multiplicity was achieved in [20,27,28]. The proposed approach involves the transition to corresponding multiple stochastic integrals with their subsequent representation by multiple series with random coefficients. In this case, the expansion of the deterministic functions of several variables into generalized multiple Fourier series is used. Most of the attention is paid to Legendre polynomials and trigonometric functions as the basis to represent the iterated stochastic integrals of arbitrary multiplicity [29], but Walsh and Haar functions are also used to represent the iterated stochastic integrals of a second multiplicity [20].

This paper proposes exact and approximate representations of iterated Itô and Stratonovich stochastic integrals using the spectral method [30]. These representations are based on matrix expressions that use several spectral characteristics of functions, operators, and functionals. For iterated Itô stochastic integrals, it avoids relations with the indicators [20] or the Wick product [31] of random variables that correspond to Wiener processes. In this context, iterated Itô and Stratonovich stochastic integrals are considered earlier with the simplest weight functions [32,33,34]. Here, we propose a general case of weight functions and consider mixed-type iterated stochastic integrals of arbitrary multiplicity. In addition to theoretical results, this paper contains the results of numerical experiments.

2. Itô Stochastic Differential Equations and Taylor–Itô Expansion

We consider the nonlinear Itô stochastic differential equation as follows [35]:

$$dX\left(t\right)=f\left(t,X\left(t\right)\right)dt+\sigma \left(t,X\left(t\right)\right)dW\left(t\right),\hspace{1em}X\left(t_0\right)=X_0,$$

(1)

where $X(·)$ is the n-dimensional random process, $t\in \mathbb{T}=[t_0,T]$, $W(·)$ is the s-dimensional Wiener process with independent components, $X_0$ is the n-dimensional random vector, $f(·):\mathbb{T}\times {\mathbb{R}}^n\to {\mathbb{R}}^n$ is the given vector function, and $\sigma (·):\mathbb{T}\times {\mathbb{R}}^n\to {\mathbb{R}}^{n\times s}$ is the given matrix function, $\mathbb{R}=(-\infty ,+\infty )$. The random vector $X_0$ and the Wiener process $W(·)$ are independent.

As a rule [35], it suffices to assume that $f(·)$ and $\sigma (·)$ are measurable functions, and they satisfy both the linear growth condition and the Lipschitz condition with respect to the second argument. However, we additionally assumed that $f(·)$ and $\sigma (·)$ are sufficiently smooth functions. Thus, all of the derivatives of these functions appearing in the Taylor–Itô expansion exist [17]:

$$\begin{array}{c}X(\vartheta +h)=X\left(\vartheta \right)+hf\left(\vartheta ,X\left(\vartheta \right)\right)+\sum _{j_1=1}^s{\sigma }_{\ast j_1}\left(\vartheta ,X\left(\vartheta \right)\right){\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\mathbb{k}(·)\hfill \\ +\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)+\frac{h^2}{2} {\mathcal{A}}^{\ast }f\left(\vartheta ,X\left(\vartheta \right)\right)\hfill \\ +\sum _{j_1=1}^s\left[{\mathcal{A}}^{\ast }{\sigma }_{\ast j_1}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1\right)}\mathbb{k}(·)+{\mathcal{L}}_{j_1}f\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)\right]\hfill \\ +\sum _{j_1,j_2,j_3=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}{\sigma }_{\ast j_3}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3\right)}\mathbb{k}(·)+\sum _{j_1,j_2=1}^s{\mathcal{A}}^{\ast }{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1{j}_2\right)}\mathbb{k}(·)\hfill \\ +\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\mathcal{A}}^{\ast }{\sigma }_{\ast j_2}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0j_2\right)}\mathbb{k}(·)+\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}f\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_{2}0\right)}\mathbb{k}(·)\hfill \\ +\sum _{j_1,j_2,j_3,j_4=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}{\mathcal{L}}_{j_3}{\sigma }_{\ast j_4}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3{j}_4\right)}\mathbb{k}(·)+\dots ,\hfill \end{array}$$

(2)

where $\vartheta ⩾t_0$, $h>0$, and $\vartheta +h⩽T$.

Expansion (2) uses the following notations ($\varphi (·):\mathbb{T}\times {\mathbb{R}}^n\to \mathbb{R}$):

$$\begin{array}{cc}\hfill {\mathcal{A}}^{\ast }\varphi (t,x)& =\sum _{i=1}^n{f}_i(t,x)\frac{\partial \varphi (t,x)}{\partial x_i}+\frac{1}{2}\sum _{i,j=1}^n\left[\sum _{l=1}^s{\sigma }_{il}(t,x){\sigma }_{jl}(t,x)\right]\frac{{\partial }^2\varphi (t,x)}{\partial x_i\partial x_j},\hfill \\ & \hspace{1em}{\mathcal{L}}_j\varphi (t,x)=\sum _{i=1}^n\frac{\partial \varphi (t,x)}{\partial x_i}{\sigma }_{ij}(t,x),\hspace{1em}j=1,\dots ,s,\hfill \end{array}$$

where ${\sigma }_{\ast j}(·)$ is the jth column of the matrix function $\sigma (·)$.

Further, ${}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}\mathbb{k}(·)$ is the iterated Itô stochastic integral, i.e.,

$$\begin{array}{c}\hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}\mathbb{k}(·)={\int }_{\vartheta }^{\vartheta +h}\dots {\int }_{\vartheta }^{t_3}{\int }_{\vartheta }^{t_2}d{W}_{j_1}\left(t_1\right)d{W}_{j_2}\left(t_2\right)\dots d{W}_{j_k}\left(t_k\right),\end{array}$$

(3)

where $j_1,\dots ,j_k\in \{0,1,\dots ,s\}$, k is the integral multiplicity, $W_0\left(t\right)=t$, and $W_1(·),\dots ,W_s(·)$ are independent components of the s-dimensional Wiener process $W(·)$.

The notation used for the iterated Itô stochastic integral (3) is based on the fact that it can be represented as a particular case of the multiple Itô stochastic integral [20]. In fact, ${}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}$ is the linear operator that associates the function with the multiple Itô stochastic integral for this function. The number of function arguments should be the same as the integral multiplicity. In this case, the linear operator ${}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}$ acts on the function

$$\mathbb{k}(t_1,\dots ,t_k)=1(t_k-t_{k-1})\dots 1(t_2-t_1)=\left\{\begin{array}{cc}1\hfill & \mathrm{for} t_1<\dots <t_k\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(4)

where

$$1(t-\tau )=\left\{\begin{array}{cc}1\hfill & \mathrm{for} t>\tau \hfill \\ 0\hfill & \mathrm{for} t⩽\tau ,\hfill \end{array}\right.$$

and $\mathbb{k}\left(t\right)\equiv 1$ for $k=1$.

However, below we consider a more general iterated Itô stochastic integral

$$\begin{array}{cc}\hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)& ={\int }_{\vartheta }^{\vartheta +h}\dots {\int }_{\vartheta }^{t_3}{\int }_{\vartheta }^{t_2}{\psi }_1\left(t_1\right){\psi }_2\left(t_2\right)\dots {\psi }_k\left(t_k\right)\hfill \\ & \times d{W}_{j_1}\left(t_1\right)d{W}_{j_2}\left(t_2\right)\dots d{W}_{j_k}\left(t_k\right),\hfill \end{array}$$

(5)

where ${\psi }_l(·):\mathbb{T}\to \mathbb{R}$ is the bounded weight function, $l=1,\dots ,k$. This notation means that the linear operator ${}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}$ acts on the function

$$\begin{array}{cc}\hfill {\mathbb{k}}_{\psi }(t_1,\dots ,t_k)& ={\psi }_1\left(t_1\right)\dots {\psi }_k\left(t_k\right)1(t_k-t_{k-1})\dots 1(t_2-t_1)\hfill \\ & =\left\{\begin{array}{cc}{\psi }_1\left(t_1\right)\dots {\psi }_k\left(t_k\right)\hfill & t_1<\dots <t_k\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.\hfill \end{array}$$

(6)

where ${\mathbb{k}}_{\psi }\left(t\right)=\psi \left(t\right)$ is used for $k=1$.

In particular, if at least one of the values $j_1,\dots ,j_k$ is equal to zero (the corresponding integral is called the mixed-type iterated stochastic integral), then the integral (3) can be reduced to a linear combination of integrals (5) with a smaller multiplicity and the additional condition [20]

$${\psi }_l\left(t\right)={(t-\vartheta )}^{n_l},\hspace{1em}{n}_l\in \{0,1,2,\dots \},\hspace{1em}l=1,\dots ,k.$$

(7)

The function ${\mathbb{k}}_{\psi }(·)$ under condition (7) is denoted by ${\mathbb{k}}_{n_1\dots n_k}(·)$. For example,

$$\begin{array}{c}\begin{array}{cccc}\hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1\right)}\mathbb{k}(·)& ={\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}{\mathbb{k}}_1(·),\hfill & \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)& ={\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\left(h-{\mathbb{k}}_1(·)\right),\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1{j}_2\right)}\mathbb{k}(·)& ={}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}{\mathbb{k}}_{10}(·),\hfill & \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0j_2\right)}\mathbb{k}(·)& ={}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\left({\mathbb{k}}_{01}(·)-{\mathbb{k}}_{10}(·)\right),\hfill \end{array}\\ {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_{2}0\right)}\mathbb{k}(·)={}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\left(h\mathbb{k}(·)-{\mathbb{k}}_{01}(·)\right),\end{array}$$

(8)

where the number of function arguments coincides with the integral multiplicity, and it suffices to use the notation ${\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}$ instead of ${}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}$ for $k=1$.

By substituting Relations (8) between iterated Itô stochastic integrals into Formula (2), the unified Taylor–Itô expansion is obtained [20]. The difference is that Expansion (2) contains iterated stochastic integrals for functions (4) only, but some of them are mixed-type iterated stochastic integrals. In the unified expansion, there are no mixed-type integrals due to the transition to functions (6) under condition (7). These expansions are equivalent, but the latter expansion contains fewer types of iterated stochastic integrals.

Based on the Taylor–Itô expansion, a great deal of numerical methods for solving stochastic differential equations have been proposed [13,17,18,20]. To describe some of them, we should introduce a discretization of the interval $[t_0,T]$ with a given constant step size h (a variable step size can also be considered). A discrete-time approximation for the random process $X(·)$ is defined by a numerical method such as the following:

$$\begin{array}{c}{X}_i\approx X\left({\vartheta }_i\right),\hspace{1em}i=1,2,\dots ,N;\hspace{1em}{\vartheta }_{i+1}={\vartheta }_i+h,\hspace{1em}i=0,1,\dots ,N-1;\\ {\vartheta }_0=t_0,\hspace{1em}{\vartheta }_N=T;\hspace{1em}N=\frac{T-t_0}{h}.\end{array}$$

The type and the order of convergence are very important for numerical methods. The numerical method has the order of mean-square convergence p if

$$\underset{i\in \{1,\dots ,N\}}{max}{\left(\mathrm{E} |X\left({\vartheta }_i\right)-X_i{|}^2\right)}^{\frac{1}{2}}⩽C{h}^p,$$

and the numerical method has the order of strong convergence p if

$$\underset{i\in \{1,\dots ,N\}}{max}\mathrm{E} |X\left({\vartheta }_i\right)-X_i|⩽C{h}^p,$$

where $\mathrm{E}$ means the expectation, $C>0$ is a constant independent of step size h, and $h\to 0$. Note also that the second inequality follows from the first one. This can be proved using the Jensen inequality [36].

To construct a numerical method with the order of mean-square or strong convergence p, it suffices to leave only the first terms on the right-hand side of Expansion (2). For example, we can write the explicit difference schemes for the stochastic differential equation (1) as follows:

$$\begin{array}{cc}p=0.5\hspace{1em}(\mathrm{Euler}–\mathrm{Maruyama} \mathrm{method}):\hfill & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

$$\begin{array}{c}{X}_{i+1}=X_i+hf({\vartheta }_i,X_i)+\sum _{j_1=1}^s{\sigma }_{\ast j_1}({\vartheta }_i,X_i){\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1\right)}\mathbb{k}(·),\hfill \end{array}$$

(9)

$$\begin{array}{cc}p=1.0\hspace{1em}(\mathrm{Milstein}–\mathrm{type} \mathrm{method}):\hfill & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

$$\begin{array}{c}{X}_{i+1}=X_i+hf({\vartheta }_i,X_i)+\sum _{j_1=1}^s{\sigma }_{\ast j_1}({\vartheta }_i,X_i){\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1\right)}\mathbb{k}(·)+\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·),\hfill \end{array}$$

(10)

$$\begin{array}{cc}p=1.5\hspace{1em}(\mathrm{Strong} 1.5 \mathrm{order} \mathrm{method}):\hfill & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

$$\begin{array}{cc}\hfill X_{i+1}& =X_i+hf({\vartheta }_i,X_i)+\sum _{j_1=1}^s{\sigma }_{\ast j_1}({\vartheta }_i,X_i){\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1\right)}\mathbb{k}(·)\hfill \\ & +\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)+\frac{h^2}{2} {\mathcal{A}}^{\ast }f({\vartheta }_i,X_i)\hfill \\ & +\sum _{j_1=1}^s\left[{\mathcal{A}}^{\ast }{\sigma }_{\ast j_1}({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(0j_1\right)}\mathbb{k}(·)+{\mathcal{L}}_{j_1}f({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)\right]\hfill \\ & +\sum _{j_1,j_2,j_3=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}{\sigma }_{\ast j_3}({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2{j}_3\right)}\mathbb{k}(·),\hfill \end{array}$$

(11)

$$\begin{array}{cc}p=2.0\hspace{1em}(\mathrm{Strong} 2.0 \mathrm{order} \mathrm{method}):\hfill & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

$$\begin{array}{cc}\hfill X_{i+1}& =X_i+hf({\vartheta }_i,X_i)+\sum _{j_1=1}^s{\sigma }_{\ast j_1}({\vartheta }_i,X_i){\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1\right)}\mathbb{k}(·)\hfill \\ & +\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)+\frac{h^2}{2} {\mathcal{A}}^{\ast }f({\vartheta }_i,X_i)\hfill \\ & +\sum _{j_1=1}^s\left[{\mathcal{A}}^{\ast }{\sigma }_{\ast j_1}({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(0j_1\right)}\mathbb{k}(·)+{\mathcal{L}}_{j_1}f({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)\right]\hfill \\ & +\sum _{j_1,j_2,j_3=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}{\sigma }_{\ast j_3}({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2{j}_3\right)}\mathbb{k}(·)+\sum _{j_1,j_2=1}^s{\mathcal{A}}^{\ast }{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(0j_1{j}_2\right)}\mathbb{k}(·)\hfill \\ & +\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\mathcal{A}}^{\ast }{\sigma }_{\ast j_2}({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_{1}0j_2\right)}\mathbb{k}(·)+\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}f({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_{2}0\right)}\mathbb{k}(·)\hfill \\ & +\sum _{j_1,j_2,j_3,j_4=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}{\mathcal{L}}_{j_3}{\sigma }_{\ast j_4}({\vartheta }_i,X_i){}^{\mathrm{I}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2{j}_3{j}_4\right)}\mathbb{k}(·).\hfill \end{array}$$

(12)

3. Stratonovich Stochastic Differential Equations and Taylor–Stratonovich Expansion

We also consider the nonlinear Stratonovich stochastic differential equation [35]:

$$dX\left(t\right)=a\left(t,X\left(t\right)\right)dt+\sigma \left(t,X\left(t\right)\right)\circ dW\left(t\right),\hspace{1em}X\left(t_0\right)=X_0,$$

(13)

for which the vector function $a(·):\mathbb{T}\times {\mathbb{R}}^n\to {\mathbb{R}}^n$ is added to the notations for Equation (1), where the symbol ∘ is to distinguish Itô and Stratonovich differential equations and stochastic integrals. If the condition

$$f(t,x)-a(t,x)=\frac{1}{2}\sum _{l=1}^s\frac{\partial {\sigma }_{\ast l}(t,x)}{\partial x} {\sigma }_{\ast l}(t,x)$$

holds, then Equations (1) and (13) are equivalent, i.e., they define the same random process $X(·)$.

We assume that the function $a(·)$ is sufficiently smooth, i.e., all of the derivatives of this function that are required in the Taylor–Stratonovich expansion exist [17]:

$$\begin{array}{c}X(\vartheta +h)=X\left(\vartheta \right)+ha\left(\vartheta ,X\left(\vartheta \right)\right)+\sum _{j_1=1}^s{\sigma }_{\ast j_1}\left(\vartheta ,X\left(\vartheta \right)\right){\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\mathbb{k}(·)\hfill \\ \hspace{1em}+\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)+\frac{h^2}{2} {\mathcal{L}}_{0}a\left(\vartheta ,X\left(\vartheta \right)\right)\hfill \\ \hspace{1em}+\sum _{j_1=1}^s\left[{\mathcal{L}}_0{\sigma }_{\ast j_1}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1\right)}\mathbb{k}(·)+{\mathcal{L}}_{j_1}a\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)\right]\hfill \\ \hspace{1em}+\sum _{j_1,j_2,j_3=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}{\sigma }_{\ast j_3}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3\right)}\mathbb{k}(·)+\sum _{j_1,j_2=1}^s{\mathcal{L}}_0{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1{j}_2\right)}\mathbb{k}(·)\hfill \\ \hspace{1em}+\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_0{\sigma }_{\ast j_2}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0j_2\right)}\mathbb{k}(·)+\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}a\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_{2}0\right)}\mathbb{k}(·)\hfill \\ \hspace{1em}+\sum _{j_1,j_2,j_3,j_4=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}{\mathcal{L}}_{j_3}{\sigma }_{\ast j_4}\left(\vartheta ,X\left(\vartheta \right)\right){}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3{j}_4\right)}\mathbb{k}(·)+\dots ,\hfill \end{array}$$

(14)

where $\vartheta ⩾t_0$, $h>0$, and $\vartheta +h⩽T$.

The additional notation for Expansion (14) is

$${\mathcal{L}}_0\psi (t,x)=\sum _{i=1}^n{a}_i(t,x)\frac{\partial \psi (t,x)}{\partial x_i},$$

and ${}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}\mathbb{k}(·)$ is the iterated Stratonovich stochastic integral

$$\begin{array}{c}\hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}\mathbb{k}(·)={\int }_{\vartheta }^{\vartheta +h}\dots {\int }_{\vartheta }^{t_3}{\int }_{\vartheta }^{t_2}d{W}_{j_1}\left(t_1\right)\circ d{W}_{j_2}\left(t_2\right)\circ \dots \circ d{W}_{j_k}\left(t_k\right),\end{array}$$

(15)

where ${}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}$ is the linear operator that associates the function with the multiple Stratonovich stochastic integral [20] for this function, and $\mathbb{k}(·)$ is defined by Formula (4).

As for iterated Itô stochastic integrals, it is useful to consider the more general iterated Stratonovich stochastic integral, i.e.,

$$\begin{array}{cc}\hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)& ={\int }_{\vartheta }^{\vartheta +h}\dots {\int }_{\vartheta }^{t_3}{\int }_{\vartheta }^{t_2}{\psi }_1\left(t_1\right){\psi }_2\left(t_2\right)\dots {\psi }_k\left(t_k\right)\hfill \\ & \circ d{W}_{j_1}\left(t_1\right)\circ d{W}_{j_2}\left(t_2\right)\circ \dots \circ d{W}_{j_k}\left(t_k\right),\hfill \end{array}$$

(16)

where ${\mathbb{k}}_{\psi }(·)$ is the function (6), whose definition includes the weight functions ${\psi }_l(·)$, $l=1,\dots ,k$.

For numerical methods, it suffices to restrict ourselves to functions ${\mathbb{k}}_{\psi }(·)$ under condition (7), i.e., functions ${\mathbb{k}}_{n_1\dots n_k}(·)$, since the integral (15) can be reduced to a linear combination of integrals (16) if at least one of the values $j_1,\dots ,j_k$ is equal to zero. In particular,

$$\begin{array}{c}\begin{array}{cccc}\hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1\right)}\mathbb{k}(·)& ={\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}{\mathbb{k}}_1(·),\hfill & \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)& ={\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\left(h-{\mathbb{k}}_1(·)\right),\hfill \\ \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1{j}_2\right)}\mathbb{k}(·)& ={}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}{\mathbb{k}}_{10}(·),\hfill & \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0j_2\right)}\mathbb{k}(·)& ={}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\left({\mathbb{k}}_{01}(·)-{\mathbb{k}}_{10}(·)\right),\hfill \end{array}\\ {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_{2}0\right)}\mathbb{k}(·)={}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\left(h\mathbb{k}(·)-{\mathbb{k}}_{01}(·)\right),\end{array}$$

(17)

where the number of function arguments coincides with the integral multiplicity, and we can use the notation ${\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}$ instead of ${}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}$ for $k=1$.

If Relations (17) between iterated Stratonovich stochastic integrals are substituted into Expansion (14), then the unified Taylor–Stratonovich expansion is obtained [20]. The difference is that Expansion (14) contains iterated stochastic integrals for functions (4) only, but some of them are mixed-type iterated stochastic integrals. In the unified expansion, there are no mixed-type integrals due to the transition to functions (6) under condition (7). These expansions are equivalent, but the latter expansion contains fewer types of iterated stochastic integrals.

Numerical methods based on the Taylor–Stratonovich expansion for solving stochastic differential equations can be proposed using the approach mentioned in the previous section [13,17,18,20].

To construct a numerical method with the order of mean-square or strong convergence p, it suffices to leave only the first terms on the right-hand side of Expansion (14). In particular, for the stochastic differential equation (13), we have the following explicit difference schemes:

$$\begin{array}{cc}p=1.0\hspace{1em}(\mathrm{Milstein}–\mathrm{type} \mathrm{method}):\hfill & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

$$\begin{array}{c}{X}_{i+1}=X_i+ha({\vartheta }_i,X_i)+\sum _{j_1=1}^s{\sigma }_{\ast j_1}({\vartheta }_i,X_i){\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1\right)}\mathbb{k}(·)+\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·),\hfill \end{array}$$

(18)

$$\begin{array}{cc}p=1.5\hspace{1em}(\mathrm{Strong} 1.5 \mathrm{order} \mathrm{method}):\hfill & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

$$\begin{array}{cc}\hfill X_{i+1}& =X_i+ha({\vartheta }_i,X_i)+\sum _{j_1=1}^s{\sigma }_{\ast j_1}({\vartheta }_i,X_i){\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1\right)}\mathbb{k}(·)\hfill \\ & +\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)+\frac{h^2}{2} {\mathcal{A}}^{\ast }f({\vartheta }_i,X_i)\hfill \\ & +\sum _{j_1=1}^s\left[{\mathcal{L}}_0{\sigma }_{\ast j_1}({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(0j_1\right)}\mathbb{k}(·)+{\mathcal{L}}_{j_1}a({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)\right]\hfill \\ & +\sum _{j_1,j_2,j_3=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}{\sigma }_{\ast j_3}({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2{j}_3\right)}\mathbb{k}(·),\hfill \end{array}$$

(19)

$$\begin{array}{cc}p=2.0\hspace{1em}(\mathrm{Strong} 2.0 \mathrm{order} \mathrm{method}):\hfill & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \end{array}$$

$$\begin{array}{cc}\hfill X_{i+1}& =X_i+ha({\vartheta }_i,X_i)+\sum _{j_1=1}^s{\sigma }_{\ast j_1}({\vartheta }_i,X_i){\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1\right)}\mathbb{k}(·)\hfill \\ & +\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)+\frac{h^2}{2} {\mathcal{L}}_{0}a({\vartheta }_i,X_i)\hfill \\ & +\sum _{j_1=1}^s\left[{\mathcal{L}}_0{\sigma }_{\ast j_1}({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(0j_1\right)}\mathbb{k}(·)+{\mathcal{L}}_{j_1}a({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)\right]\hfill \\ & +\sum _{j_1,j_2,j_3=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}{\sigma }_{\ast j_3}({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2{j}_3\right)}\mathbb{k}(·)+\sum _{j_1,j_2=1}^s{\mathcal{L}}_0{\mathcal{L}}_{j_1}{\sigma }_{\ast j_2}({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(0j_1{j}_2\right)}\mathbb{k}(·)\hfill \\ & +\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_0{\sigma }_{\ast j_2}({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_{1}0j_2\right)}\mathbb{k}(·)+\sum _{j_1,j_2=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}a({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_{2}0\right)}\mathbb{k}(·)\hfill \\ & +\sum _{j_1,j_2,j_3,j_4=1}^s{\mathcal{L}}_{j_1}{\mathcal{L}}_{j_2}{\mathcal{L}}_{j_3}{\sigma }_{\ast j_4}({\vartheta }_i,X_i){}^{\mathrm{S}}{\mathcal{J}}_{[{\vartheta }_i,{\vartheta }_i+h]}^{W\left(j_1{j}_2{j}_3{j}_4\right)}\mathbb{k}(·),\hfill \end{array}$$

(20)

where the leading term, which does not contain stochastic integrals, should be taken from Expansion (2) for the odd $r=2p$.

4. Fundamentals of the Spectral Method

To obtain the main result, it is proposed to apply the spectral method [33,34]. Here, it is described rather briefly, where only the notations and properties required below are given.

It is well known that all separable Hilbert spaces are isomorphic, i.e., it is possible to establish a one-to-one correspondence between the elements from separable Hilbert spaces. In this sense, the main space is the square-summable sequence space ${\ell }_2$[37]. It is convenient to represent an element from ${\ell }_2$ as an infinite column matrix in order to apply the matrix algebra techniques to it.

For example, let $L_2\left(\mathbb{H}\right)$ be the space of square-integrable functions $x(·):\mathbb{H}\to \mathbb{R}$, $\mathbb{H}=[\vartheta ,\vartheta +h]$, with the standard inner product ${(·,·)}_{L_2\left(\mathbb{H}\right)}$, and let ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$ be the orthonormal basis of $L_2\left(\mathbb{H}\right)$. Then, any function $x(·)\in L_2\left(\mathbb{H}\right)$ can be associated with the following infinite column matrix:

$$X=\left[\begin{array}{c}{X}_0\\ X_1\\ X_2\\ ⋮\end{array}\right],$$

where

$$X_i={\left(q(i,·),x(·)\right)}_{L_2\left(\mathbb{H}\right)}={\int }_{\mathbb{H}}q(i,t)x\left(t\right)dt,\hspace{1em}i=0,1,2,\dots$$

(21)

The infinite column matrix X is called the spectral characteristic of the function $x(·)$ with respect to the basis ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$. Further, it is assumed that $q(i,·)\in L_{\infty }\left(\mathbb{H}\right)$, $i=0,1,2,\dots$, where $L_{\infty }\left(\mathbb{H}\right)$ denotes the space of measurable and bounded functions $x(·):\mathbb{H}\to \mathbb{R}$.

Linear transformations of the element from the original Hilbert space are reduced to linear transformations of the corresponding element from ${\ell }_2$. If the original Hilbert space is $L_2\left(\mathbb{H}\right)$, then a linear transformation of $x(·)\in L_2\left(\mathbb{H}\right)$ corresponds to a linear transformation of its spectral characteristic X. Obviously, such transformations should be given by infinite matrices. Multilinear transformations of an ordered set of spectral characteristics are more complex, and they should be given by multidimensional infinite matrices with a dimension greater than 2. For example, a bilinear transformation is given by a three-dimensional infinite matrix. The elements of these infinite matrices generally depend on the basis ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$.

Let $\mathcal{R}$ be a multilinear operator such that

$$y(·)=\mathcal{R}(x_1(·),\dots ,x_k(·)),\hspace{1em}{x}_1(·),\dots ,x_k(·),y(·)\in L_2\left(\mathbb{H}\right),$$

then elements of the infinite $(k+1)$-dimensional matrix R are defined as follows:

$$\begin{array}{c}{R}_{i_1{i}_2\dots i_{k+1}}={\left(q(i_1,·),y_{i_2\dots i_{k+1}}(·)\right)}_{L_2\left(\mathbb{H}\right)}={\int }_{\mathbb{H}}q(i_1,t)y_{i_2\dots i_{k+1}}\left(t\right)dt,\\ i_1,i_2,\dots ,i_{k+1}=0,1,2,\dots ,\end{array}$$

(22)

where $y_{i_2\dots i_{k+1}}(·)=\mathcal{R}(q(i_2,·),\dots ,q(i_{k+1},·))$. The infinite matrix R is called the spectral characteristic of the multilinear operator $\mathcal{R}$ with respect to the basis ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$.

Linear functionals defined on the Hilbert space correspond to linear functionals that are defined on ${\ell }_2$; hence, it is also convenient to represent them by infinite column matrices. For multilinear functionals, it is proposed to use multidimensional infinite matrices with a dimension greater than 1.

Let $\mathcal{F}$ be a multilinear functional such that

$$\mathcal{F}(x_1(·),\dots ,x_k(·))\in \mathbb{R},\hspace{1em}{x}_1(·),\dots ,x_k(·)\in L_2\left(\mathbb{H}\right),$$

then elements of the infinite k-dimensional matrix F are given by the following equation

$$F_{i_1\dots i_k}=\mathcal{F}(q(i_1,·),\dots ,q(i_k,·)),\hspace{1em}{i}_1,\dots ,i_k=0,1,2,\dots ,$$

(23)

and the infinite matrix or the column matrix F is called the spectral characteristic of the multilinear functional $\mathcal{F}$ with respect to the basis ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$.

Linear and multilinear operators and functionals need not be defined for all functions from $L_2\left(\mathbb{H}\right)$. It suffices to consider them on some suitable linear subspaces.

For brevity, we introduce the notation $\mathbb{S}$ for the spectral transform to indicate the correspondence between functions, operators, functionals, and their spectral characteristics. Thus,

$$X=\mathbb{S}\left[x(·)\right],\hspace{1em}R=\mathbb{S}\left[\mathcal{R}\right],\hspace{1em}F=\mathbb{S}\left[\mathcal{F}\right].$$

By appropriately defining the multiplication of multidimensional matrices, we can reduce the linear and multilinear transformations of functions to operations with their spectral characteristics when using the matrix algebra techniques. An important point to consider is that algebraic operations should be carried out with infinite matrices, consequently, the problem of series convergence arises when we need to multiply matrices. However, it is solved quite simply if we consider only compact or bounded operators, as well as bounded functionals.

Next, we give some examples of the spectral characteristics of functions, linear and bilinear operators, and linear functionals, which are used below.

Consider the function $\mathbb{k}\left(t\right)\equiv 1$, and indicate the elements of its spectral characteristic ${\mathcal{V}}_0$ according to Equation (21):

$${\mathcal{V}}_0=\mathbb{S}\left[\mathbb{k}(·)\right],\hspace{1em}{\left({\mathcal{V}}_0\right)}_i={\left(q(i,·),\mathbb{k}(·)\right)}_{L_2\left(\mathbb{H}\right)}={\int }_{\mathbb{H}}q(i,t)dt,\hspace{1em}i=0,1,2,\dots$$

The spectral characteristic F of the function ${\mathbb{k}}_1\left(t\right)=t-\vartheta$ is defined similarly, i.e.,

$$F=\mathbb{S}\left[{\mathbb{k}}_1(·)\right],\hspace{1em}{F}_i={\left(q(i,·),{\mathbb{k}}_1(·)\right)}_{L_2\left(\mathbb{H}\right)}={\int }_{\mathbb{H}}q(i,t)(t-\vartheta )dt,\hspace{1em}i=0,1,2,\dots ,$$

where $\vartheta$ is a parameter (which is omitted for notational simplicity).

The infinite column matrix ${\mathcal{V}}_0$ is also the spectral characteristic of the linear functional ${\mathcal{J}}_{\mathbb{H}}$, which associates the function $x(·)\in L_2\left(\mathbb{H}\right)$ with its integral (bounded functional [37]):

$${\mathcal{J}}_{\mathbb{H}}x(·)={\int }_{\mathbb{H}}x\left(t\right)dt,$$

and this follows from Equation (23) under condition $k=1$, i.e.,

$${\mathcal{V}}_0=\mathbb{S}\left[{\mathcal{J}}_{\mathbb{H}}\right],\hspace{1em}{\left({\mathcal{V}}_0\right)}_i={\mathcal{J}}_{\mathbb{H}}q(i,·)={\int }_{\mathbb{H}}q(i,t)dt,\hspace{1em}i=0,1,2,\dots ,$$

then

$${\mathcal{J}}_{\mathbb{H}}x(·)={\left(\mathbb{S}\left[{\mathcal{J}}_{\mathbb{H}}\right]\right)}^{\mathrm{T}}\mathbb{S}\left[x(·)\right]={\mathcal{V}}_0^{\mathrm{T}}X,$$

where ${[ · ]}^{\mathrm{T}}$ means the matrix transposition. So, $\mathbb{S}\left[{\mathcal{J}}_{\mathbb{H}}\right]=\mathbb{S}\left[\mathbb{k}(·)\right]$. For example,

$${\mathcal{J}}_{\mathbb{H}}\mathbb{k}(·)={\int }_{\mathbb{H}}dt=h={\mathcal{V}}_0^{\mathrm{T}}{\mathcal{V}}_0\hspace{1em}\mathrm{and}\hspace{1em}{\mathcal{J}}_{\mathbb{H}}{\mathbb{k}}_1(·)={\int }_{\mathbb{H}}(t-\vartheta )dt=\frac{h^2}{2}={\mathcal{V}}_0^{\mathrm{T}}F.$$

As examples of linear operators, consider the integration operator ${\mathcal{D}}^{-1}$ (compact operator [37]) and the multiplication operator ${\mathcal{A}}_{\psi }$ with bounded multiplier $\psi (·)$ (bounded operator [38]):

$${\mathcal{D}}^{-1}x(·)={\int }_{\vartheta }^{(·)}x\left(\tau \right)d\tau ,\hspace{1em}{\mathcal{A}}_{\psi }x(·)=\psi (·)x(·),\hspace{1em}x(·)\in L_2\left(\mathbb{H}\right),\hspace{1em}\psi (·)\in L_{\infty }\left(\mathbb{H}\right),$$

where elements of their spectral characteristics satisfy Equation (22) under condition $k=1$:

$$\begin{array}{c}\begin{array}{cccc}\hfill P^{-1}& =\mathbb{S}\left[{\mathcal{D}}^{-1}\right],\hfill & \hfill P_{i_1{i}_2}^{-1}& ={\left(q(i_1,·),{\mathcal{D}}^{-1}q(i_2,·)\right)}_{L_2\left(\mathbb{H}\right)}={\int }_{\mathbb{H}}q(i_1,t){\int }_{\vartheta }^{t}q(i_2,\tau )d\tau dt,\hfill \\ \hfill \mathrm{\Psi }& =\mathbb{S}\left[{\mathcal{A}}_{\psi }\right],\hfill & \hfill {\mathrm{\Psi }}_{i_1{i}_2}& ={\left(q(i_1,·),{\mathcal{A}}_{\psi }q(i_2,·)\right)}_{L_2\left(\mathbb{H}\right)}={\int }_{\mathbb{H}}\psi \left(t\right)q(i_1,t)q(i_2,t)dt,\hfill \end{array}\\ i_1,i_2=0,1,2,\dots ,\end{array}$$

then

$$\mathbb{S}\left[{\mathcal{D}}^{-1}x(·)\right]=\mathbb{S}\left[{\mathcal{D}}^{-1}\right]\mathbb{S}\left[x(·)\right]=P^{-1}X\hspace{1em}\mathrm{and}\hspace{1em}\mathbb{S}\left[{\mathcal{A}}_{\psi }x(·)\right]=\mathbb{S}\left[{\mathcal{A}}_{\psi }\right]\mathbb{S}\left[x(·)\right]=\mathsf{\Psi }X.$$

If $\psi \left(t\right)={\mathbb{k}}_1\left(t\right)=t-\vartheta$, then the spectral characteristic of the multiplication operator ${\mathcal{A}}_{{\mathbb{k}}_1}$ is denoted by A. If $\psi \left(t\right)={\mathbb{k}}_n\left(t\right)={(t-\vartheta )}^n$ for $n\in \{0,1,2,\dots \}$, then it is natural to use the notation $A^n$ for the spectral characteristic of the multiplication operator ${\mathcal{A}}_{{\mathbb{k}}_n}$. Here, $\vartheta$ is also a parameter.

Another example is related to the bilinear multiplication operator $\mathcal{M}$:

$$\mathcal{M}(x(·),y(·))=x(·)y(·),\hspace{1em}x(·)\in L_2\left(\mathbb{H}\right),\hspace{1em}y(·)\in L_{\infty }\left(\mathbb{H}\right),$$

and we apply Equation (22) under condition $k=2$ to find the elements of its spectral characteristic:

$$\begin{array}{cc}\hfill V=\mathbb{S}\left[\mathcal{M}\right],\hspace{1em}{V}_{i_1{i}_2{i}_3}& ={\left(q(i_1,·),\mathcal{M}(q(i_2,·),q(i_3,·))\right)}_{L_2\left(\mathbb{H}\right)}\hfill \\ & ={\int }_{\mathbb{H}}q(i_1,t)q(i_2,t)q(i_3,t)dt,\hspace{1em}{i}_1,i_2,i_3=0,1,2,\dots ,\hfill \end{array}$$

then

$$\mathbb{S}\left[\mathcal{M}\left(x\right(·),y(·\left)\right)\right]=\left(\mathbb{S}\left[\mathcal{M}\right]\mathbb{S}\left[y(·)\right]\right)\mathbb{S}\left[x(·)\right]=\left(VY\right)X.$$

The right-hand side of the latter equality contains the product of the infinite three-dimensional matrix and the infinite column matrix. The result of this operation is the infinite matrix:

$$M=VY,\hspace{1em}{M}_{i_1{i}_2}=\sum _{i_3=0}^{\infty }{V}_{i_1{i}_2{i}_3}{Y}_{i_3},\hspace{1em}{i}_1,i_2=0,1,2,\dots ,$$

and other matrix operations, such as the multiplication of the infinite matrix and the infinite column matrix, are standard.

Additionally, we should highlight that if $\tilde{\mathrm{\Psi }}=\mathbb{S}\left[\psi (·)\right]$ and $\mathrm{\Psi }=\mathbb{S}\left[{\mathcal{A}}_{\psi }\right]$, then $\mathrm{\Psi }=V\tilde{\mathrm{\Psi }}$, since for any $x(·)\in L_2\left(\mathbb{H}\right)$ we have

$$\mathcal{M}(x(·),\psi (·))={\mathcal{A}}_{\psi }x(·)\hspace{1em}\mathrm{and}\hspace{1em}\left(V\tilde{\mathrm{\Psi }}\right)X=\mathrm{\Psi }X,$$

for example, $E=V{\mathcal{V}}_0$ and $A=VF$, where E is the infinite identity matrix (E is the spectral characteristic of the identity operator $\mathcal{I}$).

The spectral method is convenient because operations with functions are reduced to operations with their spectral characteristics. For example,

$${\mathbb{k}}_1(·)={\mathcal{D}}^{-1}\mathbb{k}(·)\hspace{1em}⟺\hspace{1em}F=P^{-1}{\mathcal{V}}_0\hspace{1em}\hspace{1em}\mathrm{or}\hspace{1em}\hspace{1em}{\mathbb{k}}_1(·)={\mathcal{A}}_{{\mathbb{k}}_1}\mathbb{k}(·)\hspace{1em}⟺\hspace{1em}F=A{\mathcal{V}}_0,$$

and

$${\mathcal{J}}_{\mathbb{H}}{\mathbb{k}}_1(·)={\mathcal{V}}_0^{\mathrm{T}}F={\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_0={\mathcal{V}}_0^{\mathrm{T}}A{\mathcal{V}}_0=\frac{h^2}{2}.$$

Similar relations can also be written for random processes, as well as for random linear and multilinear operators and functionals. In this case, it is proposed to use the same basis ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$. This approach leads to random spectral characteristics, and, here, it is natural to consider random processes, which have finite second moments and realizations that belong to $L_2\left(\mathbb{H}\right)$ with probability 1. We use the notation ${\mathcal{L}}_2\left(\mathbb{H}\right)$ for the space of such random processes. Then, Equation (21) can be used to determine the elements of spectral characteristics of random processes from ${\mathcal{L}}_2\left(\mathbb{H}\right)$. For random linear and multilinear operators and functionals, we use Equations (22) and (23).

Consider the example of the random linear functional ${\mathcal{J}}_{\mathbb{H}}^W$ that associates the function $x(·)\in L_2\left(\mathbb{H}\right)$ with its stochastic integral over the Wiener process $W(·)$:

$${\mathcal{J}}_{\mathbb{H}}^{W}x(·)={\int }_{\mathbb{H}}x\left(t\right)dW\left(t\right).$$

To determine the elements of its spectral characteristic $\mathcal{V}$, we apply Equation (23) under condition $k=1$, i.e.,

$$\mathcal{V}=\mathbb{S}\left[{\mathcal{J}}_{\mathbb{H}}^W\right],\hspace{1em}{\mathcal{V}}_i={\mathcal{J}}_{\mathbb{H}}^{W}q(i,·)={\int }_{\mathbb{H}}q(i,t)dW\left(t\right),\hspace{1em}i=0,1,2,\dots$$

According to stochastic integral properties, ${\mathcal{V}}_i$ are independent random variables that have a standard normal distribution (the normal distribution law is a consequence of the integration over the Gaussian random process, and the independence follows from the Itô isometry) [35,36]. Then

$${\mathcal{J}}_{\mathbb{H}}^{W}x(·)={\left(\mathbb{S}\left[{\mathcal{J}}_{\mathbb{H}}^W\right]\right)}^{\mathrm{T}}\mathbb{S}\left[x(·)\right]={\mathcal{V}}^{\mathrm{T}}X,$$

for example,

$${\mathcal{J}}_{\mathbb{H}}^W\mathbb{k}(·)={\int }_{\mathbb{H}}dW\left(t\right)={\mathcal{V}}^{\mathrm{T}}{\mathcal{V}}_0\hspace{1em}\mathrm{and}\hspace{1em}{\mathcal{J}}_{\mathbb{H}}^W{\mathbb{k}}_1(·)={\int }_{\mathbb{H}}(t-\vartheta )dW\left(t\right)={\mathcal{V}}^{\mathrm{T}}F.$$

As an example of the random linear operator, consider the stochastic integration operator ${\mathcal{D}}_W^{-1}$ over the Wiener process $W(·)$:

$${\mathcal{D}}_W^{-1}x(·)={\int }_{\vartheta }^{(·)}x\left(\tau \right)dW\left(\tau \right),\hspace{1em}x(·)\in L_2\left(\mathbb{H}\right),$$

where Equation (22) is applied under condition $k=1$ to obtain the elements of its spectral characteristic:

$$\begin{array}{c}\begin{array}{cccc}\hfill P^{-1,\mathcal{V}}& =\mathbb{S}\left[{\mathcal{D}}^{-1}\right],\hfill & \hfill P_{i_1{i}_2}^{-1,\mathcal{V}}& ={\left(q(i_1,·),{\mathcal{D}}_W^{-1}q(i_2,·)\right)}_{L_2\left(\mathbb{H}\right)}={\int }_{\mathbb{H}}q(i_1,t){\int }_{\vartheta }^{t}q(i_2,\tau )dW\left(\tau \right)dt,\hfill \end{array}\\ i_1,i_2=0,1,2,\dots ,\end{array}$$

then

$$\mathbb{S}\left[{\mathcal{D}}_W^{-1}x(·)\right]=\mathbb{S}\left[{\mathcal{D}}_W^{-1}\right]\mathbb{S}\left[x(·)\right]=P^{-1,\mathcal{V}}X,$$

where $\mathcal{V}$ corresponds to the spectral characteristic of the random linear functional ${\mathcal{J}}_{\mathbb{H}}^W$.

It is known that the Wiener process $W(·)$ is not differentiable in the ordinary sense. However, in the generalized sense, it is admissible to consider its derivative $V(·)$, which is called Gaussian white noise [35]. More strictly, Gaussian white noise is the generalized random process, i.e., the random linear functional, and this is precisely the functional ${\mathcal{J}}_{\mathbb{H}}^W$. Therefore, the spectral characteristic $\mathcal{V}$ is also called the spectral characteristic of Gaussian white noise $V(·)$, i.e., $\mathbb{S}\left[{\mathcal{J}}_{\mathbb{H}}^W\right]=\mathbb{S}\left[V(·)\right]$. In [33,34], the spectral characteristic $\mathcal{W}$ of the Wiener process $W(·)$ is proposed in the form $\mathcal{W}=P^{-1}\mathcal{V}$.

The random linear operator ${\mathcal{D}}_W^{-1}$ and the random linear functional ${\mathcal{J}}_{\mathbb{H}}^W$ can also act on the elements from ${\mathcal{L}}_2\left(\mathbb{H}\right)$, but under the condition that stochastic integrals are understood as Stratonovich stochastic integrals. This is relevant as it is this that only provides the ordinary integration rules and the possibility of applying the spectral method. Corresponding random linear operators and functionals are denoted by ${}^{\mathrm{S}}{\mathcal{D}}_W^{-1}$ and ${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^W$. In fact, ${}^{\mathrm{S}}{\mathcal{D}}_W^{-1}={\mathcal{D}}_W^{-1}$ and ${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^W={\mathcal{J}}_{\mathbb{H}}^W$.

For Itô stochastic integrals, a preliminary transformation to Stratonovich stochastic integrals is required [17], after which we can apply the spectral method. Corresponding random linear operators and functionals are denoted by ${}^{\mathrm{I}}{\mathcal{D}}_W^{-1}$ and ${}^{\mathrm{I}}{\mathcal{J}}_{\mathbb{H}}^W$. In general, they differ from ${}^{\mathrm{S}}{\mathcal{D}}_W^{-1}$ and ${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^W$, where, for example, ${}^{\mathrm{S}}{\mathcal{D}}_W^{-1}W(·)\ne {}^{\mathrm{I}}{\mathcal{D}}_W^{-1}W(·)$ and ${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W}W(·)\ne {}^{\mathrm{I}}{\mathcal{J}}_{\mathbb{H}}^{W}W(·)$.

5. Spectral Representation of Iterated Stochastic Integrals

In this section, we formulate the main result related to the spectral representations of iterated stochastic integrals. Since it is more natural to apply the spectral method to Stratonovich stochastic integrals, we start with their representation.

Theorem 1.

Let ${\mathcal{V}}_0$ be the spectral characteristic of the function $\mathbb{k}\left(t\right)\equiv 1$, and let ${\mathcal{V}}_1,\dots ,{\mathcal{V}}_s$ be the spectral characteristics of Gaussian white noises $V_1(·),\dots ,V_s(·)$, which correspond to the Wiener processes $W_1(·),\dots ,W_s(·)$. Let $P^{-1}$ be the spectral characteristic of the integration operator, and let V be the spectral characteristic of the bilinear multiplication operator. Moreover, let ${\mathrm{\Psi }}_1,\dots ,{\mathrm{\Psi }}_k$ be the spectral characteristics of multiplication operators with multipliers ${\psi }_1(·),\dots ,{\psi }_k(·)$. The spectral characteristics ${\mathcal{V}}_0$, $P^{-1}$, V, ${\mathrm{\Psi }}_1,\dots ,{\mathrm{\Psi }}_k$ are defined with respect to the basis ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$. Then, the iterated Stratonovich stochastic integral (16) is represented by relations

$$\begin{array}{c}{}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)={\mathcal{V}}_{j_k}^{\mathrm{T}}{\mathrm{\Psi }}_k{\mathcal{X}}_{k-1},\\ {\mathcal{X}}_l=P^{-1}{\mathrm{\Psi }}_l\left(V{\mathcal{V}}_{j_l}\right){\mathcal{X}}_{l-1},\hspace{1em}l=2,\dots ,k-1,\hspace{1em}{\mathcal{X}}_1=P^{-1}{\mathrm{\Psi }}_1{\mathcal{V}}_{j_1},\end{array}$$

(24)

 or in the explicit form

$${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)={\mathcal{V}}_{j_k}^{\mathrm{T}}{\mathrm{\Psi }}_k{P}^{-1}{\mathrm{\Psi }}_{k-1}\left(V{\mathcal{V}}_{j_{k-1}}\right)\dots P^{-1}{\mathrm{\Psi }}_2\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathrm{\Psi }}_1{\mathcal{V}}_{j_1}.$$

(25)

Proof of Theorem 1.

The introduction of the notation for the spectral characteristics of functions and random processes:

$${\mathcal{X}}_0={\mathcal{V}}_0=\mathbb{S}\left[\mathbb{k}(·)\right],\hspace{1em}{\mathcal{X}}_l=\mathbb{S}\left[X_l(·)\right],\hspace{1em}l=1,\dots ,k.$$

Moreover,

$${\mathcal{V}}_j=\mathbb{S}\left[V_j(·)\right],\hspace{1em}{P}^{-1,{\mathcal{V}}_j}=\mathbb{S}\left[ {}^{\mathrm{S}}{\mathcal{D}}_{W_j}^{-1}\right],\hspace{1em}{\mathrm{\Psi }}_l=\mathbb{S}\left[{\mathcal{A}}_{{\psi }_l}\right],\hspace{1em}j=1,\dots ,s,\hspace{1em}l=1,\dots ,k,$$

and the representation $P^{-1,{\mathcal{V}}_j}=P^{-1}\left(V{\mathcal{V}}_j\right)$ holds for the spectral characteristics $P^{-1,{\mathcal{V}}_j}$. It is proved in [39], but it can also be applied for $j=0$ since

$$P^{-1,{\mathcal{V}}_0}=P^{-1}\left(V{\mathcal{V}}_0\right)=P^{-1}E=P^{-1}\hspace{1em}({}^{\mathrm{S}}{\mathcal{D}}_{W_0}^{-1}={\mathcal{D}}^{-1}),$$

where E is the infinite identity matrix.

The iterated Stratonovich stochastic integral ${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)$ may be represented as the composition of simple operations including deterministic ($j_l=0$) or stochastic ($j_l\ne 0$) integrations, as well as the multiplication by weight functions ${\psi }_l(·)$, $l=1,\dots ,k$:

$${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)={}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W\left(j_k\right)}{\mathcal{A}}_{{\psi }_k}{X}_{k-1}(·),$$

(26)

where

$$X_l(·)={}^{\mathrm{S}}{\mathcal{D}}_{W_{j_l}}^{-1}{\mathcal{A}}_{{\psi }_l}{X}_{l-1}(·),\hspace{1em}l=1,\dots ,k-1,\hspace{1em}{X}_0(·)=\mathbb{k}(·).$$

(27)

If the spectral transform is applied to both sides of Equation (27), then

$$\begin{array}{cc}\hfill \mathbb{S}\left[X_l(·)\right]& =\mathbb{S}\left[ {}^{\mathrm{S}}{\mathcal{D}}_{W_{j_l}}^{-1}{\mathcal{A}}_{{\psi }_l}{X}_{l-1}(·)\right]=\mathbb{S}\left[ {}^{\mathrm{S}}{\mathcal{D}}_{W_{j_l}}^{-1}\right]\mathbb{S}\left[{\mathcal{A}}_{{\psi }_l}{X}_{l-1}(·)\right]\hfill \\ & =\mathbb{S}\left[{}^{\mathrm{S}}{\mathcal{D}}_{W_{j_l}}^{-1}\right]\mathbb{S}\left[{\mathcal{A}}_{{\psi }_l}\right]\mathbb{S}\left[X_{l-1}(·)\right].\hfill \end{array}$$

This means that

$${\mathcal{X}}_l=P^{-1,{\mathcal{V}}_{j_l}}{\mathrm{\Psi }}_l{\mathcal{X}}_{l-1}=P^{-1}\left(V{\mathcal{V}}_{j_l}\right){\mathrm{\Psi }}_l{\mathcal{X}}_{l-1}\hspace{1em}\mathrm{or}\hspace{1em}{\mathcal{X}}_l=P^{-1}{\mathrm{\Psi }}_l\left(V{\mathcal{V}}_{j_l}\right){\mathcal{X}}_{l-1},\hspace{1em}l=1,\dots ,k-1,$$

since ${\mathrm{\Psi }}_l$ and $V{\mathcal{V}}_{j_l}$ are symmetric matrices (the product of symmetric matrices is commutative), and the latter expression can be simplified under condition $l=1$:

$$\begin{array}{cc}\hfill {\mathcal{X}}_1& =P^{-1}{\mathrm{\Psi }}_1\left(V{\mathcal{V}}_{j_1}\right){\mathcal{X}}_0=P^{-1}{\mathrm{\Psi }}_1\left(V{\mathcal{V}}_{j_1}\right){\mathcal{V}}_0\hfill \\ & =P^{-1}{\mathrm{\Psi }}_1\left(V{\mathcal{V}}_0\right){\mathcal{V}}_{j_1}=P^{-1}{\mathrm{\Psi }}_{1}E{\mathcal{V}}_{j_1}=P^{-1}{\mathrm{\Psi }}_1{\mathcal{V}}_{j_1}.\hfill \end{array}$$

The last step in the proof is to represent the random linear functional ${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W\left(j_k\right)}$ as

$${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W\left(j_k\right)}{\mathcal{A}}_{{\psi }_k}{X}_{k-1}(·)={\left(\mathbb{S}\left[V_{j_k}(·)\right]\right)}^{\mathrm{T}}\mathbb{S}\left[{\mathcal{A}}_{{\psi }_k}{X}_{k-1}(·)\right]={\left(\mathbb{S}\left[V_{j_k}(·)\right]\right)}^{\mathrm{T}}\mathbb{S}\left[{\mathcal{A}}_{{\psi }_k}\right]\mathbb{S}\left[X_{k-1}(·)\right].$$

Therefore, we have the spectral analog of Equation (26):

$${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W\left(j_k\right)}{\mathcal{A}}_{{\psi }_k}{X}_{k-1}(·)={\mathcal{V}}_{j_k}^{\mathrm{T}}{\mathrm{\Psi }}_k{\mathcal{X}}_{k-1},$$

i.e., Relations (24) are proved. Finally, excluding ${\mathcal{X}}_1,\dots ,{\mathcal{X}}_{k-1}$ from them, we prove Relation (25). □

Using Theorem 1, we can write the spectral representation of iterated Stratonovich stochastic integrals from Expansion (14) (in this case, ${\mathrm{\Psi }}_1=\dots ={\mathrm{\Psi }}_k=E$):

$$\begin{array}{c}\begin{array}{cccc}\hfill {\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_1}^{\mathrm{T}}{\mathcal{V}}_0,\hfill & \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_3}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1},\hfill \\ \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1},\hfill & \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1{j}_2\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_1}\right)P^{-1}{\mathcal{V}}_0,\hfill \\ \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_1}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_0,\hfill & \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0j_2\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_0\right)P^{-1}{\mathcal{V}}_{j_1},\hfill \\ \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)& ={\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1},\hfill & \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_{2}0\right)}\mathbb{k}(·)& ={\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1},\hfill \end{array}\\ {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3{j}_4\right)}\mathbb{k}(·)={\mathcal{V}}_{j_4}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_3}\right)P^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1},\end{array}$$

(28)

where $j_1,j_2,j_3,j_4\in \{1,\dots ,s\}$.

If we use the unified Taylor–Stratonovich expansion from [20], then we need the following iterated Stratonovich stochastic integrals and their spectral representation:

$$\begin{array}{c}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}{\mathbb{k}}_1(·)={\mathcal{V}}_{j_1}^{\mathrm{T}}F,\hfill \\ {\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\left(h-{\mathbb{k}}_1(·)\right)=h{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\mathbb{k}(·)-{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}{\mathbb{k}}_1(·)=h{\mathcal{V}}_{j_1}^{\mathrm{T}}{\mathcal{V}}_0-{\mathcal{V}}_{j_1}^{\mathrm{T}}F={\mathcal{V}}_{j_1}^{\mathrm{T}}(h{\mathcal{V}}_0-F),\hfill \\ {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}{\mathbb{k}}_{10}(·)={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}A{\mathcal{V}}_{j_1},\hfill \\ {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\left({\mathbb{k}}_{01}(·)-{\mathbb{k}}_{10}(·)\right)={}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}{\mathbb{k}}_{01}(·)-{}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}{\mathbb{k}}_{10}(·)\hfill \\ \hspace{1em}={\mathcal{V}}_{j_2}^{\mathrm{T}}A{P}^{-1}{\mathcal{V}}_{j_1}-{\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}A{\mathcal{V}}_{j_1}={\mathcal{V}}_{j_2}^{\mathrm{T}}(A{P}^{-1}-P^{-1}A){\mathcal{V}}_{j_1},\hfill \\ {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\left(h\mathbb{k}(·)-{\mathbb{k}}_{01}(·)\right)=h{}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)-{}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}{\mathbb{k}}_{01}(·)\hfill \\ \hspace{1em}=h{\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1}-{\mathcal{V}}_{j_2}^{\mathrm{T}}A{P}^{-1}{\mathcal{V}}_{j_1}={\mathcal{V}}_{j_2}^{\mathrm{T}}(hE-A)P^{-1}{\mathcal{V}}_{j_1}.\hfill \end{array}$$

(29)

When comparing Relations (17), (28), and (29), we derive the following useful relations:

$$\begin{array}{c}{\mathcal{V}}_{j_1}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_0={\mathcal{V}}_{j_1}^{\mathrm{T}}F,\hspace{1em}{\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1}={\mathcal{V}}_{j_1}^{\mathrm{T}}(h{\mathcal{V}}_0-F),\hfill \\ {\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_1}\right)P^{-1}{\mathcal{V}}_0={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}A{\mathcal{V}}_{j_1},\hfill \\ {\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_0\right)P^{-1}{\mathcal{V}}_{j_1}={\mathcal{V}}_{j_2}^{\mathrm{T}}(A{P}^{-1}-P^{-1}A){\mathcal{V}}_{j_1},\hfill \\ {\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1}={\mathcal{V}}_{j_2}^{\mathrm{T}}(hE-A)P^{-1}{\mathcal{V}}_{j_1}.\hfill \end{array}$$

(30)

Moreover, we have

$${\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-2}{\mathcal{V}}_{j_1}={\mathcal{V}}_{j_2}^{\mathrm{T}}(A{P}^{-1}-P^{-1}A){\mathcal{V}}_{j_1}$$

(31)

since $P^{-1}\left(V{\mathcal{V}}_0\right)P^{-1}=P^{-1}E{P}^{-1}=P^{-2}$.

Note that Relations (30) and (31) can be proved using only the properties of the spectral characteristics ${\mathcal{V}}_0$, F, $P^{-1}$, A, and V. However, in this case, it suffices to prove them indirectly by comparing iterated Stratonovich stochastic integrals and their spectral representation.

Thus, the final form for the spectral representation of iterated stochastic integrals from Expansion (14) is

$$\begin{array}{c}\begin{array}{cccc}\hfill {\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_1}^{\mathrm{T}}{\mathcal{V}}_0,\hfill & \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_3}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1},\hfill \\ \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1},\hfill & \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1{j}_2\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}A{\mathcal{V}}_{j_1},\hfill \\ \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_1}^{\mathrm{T}}F,\hfill & \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0j_2\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}(A{P}^{-1}-P^{-1}A){\mathcal{V}}_{j_1},\hfill \\ \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_1}^{\mathrm{T}}(h{\mathcal{V}}_0-F),\hfill & \hfill {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_{2}0\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}(hE-A)P^{-1}{\mathcal{V}}_{j_1},\hfill \end{array}\\ {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3{j}_4\right)}\mathbb{k}(·)={\mathcal{V}}_{j_4}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_3}\right)P^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1},\end{array}$$

(32)

i.e., all the necessary iterated stochastic integrals to be simulated for the implementation of numerical methods (18)–(20) for solving the stochastic differential equation (13) are expressed using algebraic operations with column matrices ${\mathcal{V}}_0,{\mathcal{V}}_1,\dots ,{\mathcal{V}}_s,F$, matrices $P^{-1}$ and A, and the three-dimensional matrix V. The spectral characteristics ${\mathcal{V}}_1,\dots ,{\mathcal{V}}_s$ are random, but other spectral characteristics are deterministic.

An important point is that the spectral characteristics ${\mathcal{V}}_0$, F, $P^{-1}$, A, and V depend on the values $\vartheta$ and h. However, the property

$${}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}{\mathbb{k}}_{n_1\dots n_k}(·)\stackrel{d}{=}{h}^{k/2+{\sum }_{l=1}^k({\delta }_{j_{l}0}/2+n_l)}·{}^{\mathrm{S}}{\mathcal{J}}_{[0,1]}^{W(j_1\dots j_k)}{\mathbb{k}}_{n_1\dots n_k}^{\ast }(·)$$

(33)

holds for iterated Stratonovich stochastic integrals, where $\stackrel{d}{=}$ means equality in distribution, ${\mathbb{k}}_{n_1\dots n_k}^{\ast }(·)$ is the function (6) under condition (7) with $t_0=0$, and ${\delta }_{j_{l}0}$ is the Kronecker delta, $l=1,\dots ,k$. Therefore, it suffices to find the spectral characteristics ${\mathcal{V}}_0$, F, $P^{-1}$, A, and V with respect to the basis ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$ of $L_2\left([0,1]\right)$.

Further, we consider the spectral representation of iterated Itô stochastic integrals (5).

Theorem 2.

Let the conditions of Theorem 1 hold. Then, the Itô iterated stochastic integral (5) is represented by relations

$$\begin{array}{c}{}^{\mathrm{I}}{\mathcal{J}}_{\mathbb{H}}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)=-\frac{1}{2}{\delta }_{j_{k-1}{j}_k}^{\ast }{\mathcal{V}}_0^{\mathrm{T}}{\mathrm{\Psi }}_{k-1}{\mathrm{\Psi }}_k{\mathcal{X}}_{k-2}+{\mathcal{V}}_{j_k}^{\mathrm{T}}{\mathrm{\Psi }}_k{\mathcal{X}}_{k-1},\\ {\mathcal{X}}_l=-\frac{1}{2} {\delta }_{j_{l-1}{j}_l}^{\ast }{P}^{-1}{\mathrm{\Psi }}_{l-1}{\mathrm{\Psi }}_l{\mathcal{X}}_{l-2}+P^{-1}{\mathrm{\Psi }}_l\left(V{\mathcal{V}}_{j_l}\right){\mathcal{X}}_{l-1}, l=2,\dots ,k-1,\\ {\mathcal{X}}_0={\mathcal{V}}_0,\hspace{1em}{\mathcal{X}}_1=P^{-1}{\mathrm{\Psi }}_1{\mathcal{V}}_{j_1},\end{array}$$

(34)

 where ${\delta }_{j_{l-1}{j}_l}^{\ast }={\delta }_{j_{l-1}{j}_l}(1-{\delta }_{j_{l}0})$, ${\delta }_{j_{l-1}{j}_l}$ and ${\delta }_{j_{l}0}$ are Kronecker deltas, $l=2,\dots ,k$.

Proof of Theorem 2.

Using the approach from the proof of Theorem 1, we may represent the iterated stochastic integral ${}^{\mathrm{I}}{\mathcal{J}}_{\mathbb{H}}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)$ as the composition of simple operations, including deterministic ($j_l=0$) or stochastic ($j_l\ne 0$) integrations, as well as the multiplication by weight functions ${\psi }_l(·)$, $l=1,\dots ,k$:

$${}^{\mathrm{I}}{\mathcal{J}}_{\mathbb{H}}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)={}^{\mathrm{I}}{\mathcal{J}}_{\mathbb{H}}^{W\left(j_k\right)}{\mathcal{A}}_{{\psi }_k}{X}_{k-1}(·),$$

where

$$X_l(·)={}^{\mathrm{I}}{\mathcal{D}}_{W_{j_l}}^{-1}{\mathcal{A}}_{{\psi }_l}{X}_{l-1}(·),\hspace{1em}l=1,\dots ,k-1,\hspace{1em}{X}_0\left(t\right)=\mathbb{k}\left(t\right).$$

The next step is to express ${}^{\mathrm{I}}{\mathcal{J}}_{\mathbb{H}}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)$ via Stratonovich stochastic integrals. For $l=1$, it is necessary to change the notation only since the Itô and Stratonovich stochastic integrals of deterministic functions coincide, i.e.,

$$X_1(·)={}^{\mathrm{S}}{\mathcal{D}}_{W_{j_1}}^{-1}{\mathcal{A}}_{{\psi }_1}{X}_0(·),$$

(35)

and for $l>1$, it is required to use the relationship between the Itô and Stratonovich stochastic integrals [17]:

$$X_l(·)=-\frac{{\delta }_{j_{l-1}{j}_l}^{\ast }}{2}{\mathcal{D}}^{-1}{\mathcal{A}}_{{\psi }_{l-1}}{\mathcal{A}}_{{\psi }_l}{X}_{l-2}(·)+{}^{\mathrm{S}}{\mathcal{D}}_{W_{j_l}}^{-1}{\mathcal{A}}_{{\psi }_l}{X}_{l-1}(·),\hspace{1em}l=2,\dots ,k-1,$$

(36)

and

$${}^{\mathrm{I}}{\mathcal{J}}_{\mathbb{H}}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)=-\frac{{\delta }_{j_{k-1}{j}_k}^{\ast }}{2}{\mathcal{J}}_{\mathbb{H}}{\mathcal{A}}_{{\psi }_{k-1}}{\mathcal{A}}_{{\psi }_k}{X}_{k-2}(·)+{}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W\left(j_k\right)}{\mathcal{A}}_{{\psi }_k}{X}_{k-1}(·).$$

(37)

The use of the values ${\delta }_{j_{l-1}{j}_l}^{\ast }$ instead of ${\delta }_{j_{l-1}{j}_l}$ is due to the fact that mixed-type iterated stochastic integrals are considered, i.e., not only the stochastic integration, but also the deterministic integration is allowed, in which the relationship between the Itô and Stratonovich stochastic integrals is not used.

Equation (35) is the same as for the iterated Stratonovich stochastic integral since if $l=1$, then the integration is performed for the deterministic function. Therefore,

$${\mathcal{X}}_1=P^{-1}{\mathrm{\Psi }}_1{\mathcal{V}}_{j_1}.$$

If the spectral transform is applied to both sides of Equation (36) when using the notations from the proof of Theorem 1 and the linearity property ($\mathbb{S}$ is the linear transform), then

$$\begin{array}{cc}\hfill \mathbb{S}\left[X_l(·)\right]& =\mathbb{S}\left[-\frac{{\delta }_{j_{l-1}{j}_l}^{\ast }}{2}{\mathcal{D}}^{-1}{\mathcal{A}}_{{\psi }_{l-1}}{\mathcal{A}}_{{\psi }_l}{X}_{l-2}(·)+{}^{\mathrm{S}}{\mathcal{D}}_{W_{j_l}}^{-1}{\mathcal{A}}_{{\psi }_l}{X}_{l-1}(·)\right]\hfill \\ & =-\frac{{\delta }_{j_{l-1}{j}_l}^{\ast }}{2}\mathbb{S}\left[{\mathcal{D}}^{-1}{\mathcal{A}}_{{\psi }_{l-1}}{\mathcal{A}}_{{\psi }_l}{X}_{l-2}(·)\right]+\mathbb{S}\left[{}^{\mathrm{S}}{\mathcal{D}}_{W_{j_l}}^{-1}{\mathcal{A}}_{{\psi }_l}{X}_{l-1}(·)\right].\hfill \end{array}$$

The expression for the second term on the right-hand side of the latter equality is obtained in the proof of Theorem 1, so here we can restrict ourselves to the first term only:

$$\begin{array}{c}\mathbb{S}\left[{\mathcal{D}}^{-1}{\mathcal{A}}_{{\psi }_{l-1}}{\mathcal{A}}_{{\psi }_l}{X}_{l-2}(·)\right]=\mathbb{S}\left[{\mathcal{D}}^{-1}\right]\mathbb{S}\left[{\mathcal{A}}_{{\psi }_{l-1}}{\mathcal{A}}_{{\psi }_l}{X}_{l-2}(·)\right]\hfill \\ =\mathbb{S}\left[{\mathcal{D}}^{-1}\right]\mathbb{S}\left[{\mathcal{A}}_{{\psi }_{l-1}}\right]\mathbb{S}\left[{\mathcal{A}}_{{\psi }_l}{X}_{l-2}(·)\right]=\mathbb{S}\left[{\mathcal{D}}^{-1}\right]\mathbb{S}\left[{\mathcal{A}}_{{\psi }_{l-1}}\right]\mathbb{S}\left[{\mathcal{A}}_{{\psi }_l}\right]\mathbb{S}\left[X_{l-2}(·)\right],\hfill \end{array}$$

hence

$$\begin{array}{cc}\hfill {\mathcal{X}}_l& =-\frac{{\delta }_{j_{l-1}{j}_l}^{\ast }}{2}{P}^{-1}{\mathrm{\Psi }}_{l-1}{\mathrm{\Psi }}_l{\mathcal{X}}_{l-2}+P^{-1}{\mathrm{\Psi }}_l\left(V{\mathcal{V}}_{j_l}\right){\mathcal{X}}_{l-1}\hfill \\ & =P^{-1}{\mathrm{\Psi }}_l\left[-\frac{{\delta }_{j_{l-1}{j}_l}^{\ast }}{2}{\mathrm{\Psi }}_{l-1}{\mathcal{X}}_{l-2}+\left(V{\mathcal{V}}_{j_l}\right){\mathcal{X}}_{l-1}\right],\hspace{1em}l=2,\dots ,k-1.\hfill \end{array}$$

It remains to represent the random linear functional ${}^{\mathrm{I}}{\mathcal{J}}_{\mathbb{H}}^{W\left(j_k\right)}$, but its value includes ${}^{\mathrm{S}}{\mathcal{J}}_{\mathbb{H}}^{W\left(j_k\right)}{\mathcal{A}}_{{\psi }_k}{X}_{k-1}(·)$, which is obtained in the proof of Theorem 1. Therefore, it suffices to consider the first term on the right-hand side of Relation (37):

$$\begin{array}{c}{\mathcal{J}}_{\mathbb{H}}{\mathcal{A}}_{{\psi }_{k-1}}{\mathcal{A}}_{{\psi }_k}{X}_{k-2}(·)={\left(\mathbb{S}\left[{\mathcal{J}}_{\mathbb{H}}\right]\right)}^{\mathrm{T}}\mathbb{S}\left[{\mathcal{A}}_{{\psi }_{k-1}}{\mathcal{A}}_{{\psi }_k}{X}_{k-2}(·)\right]\hfill \\ \hspace{1em}={\left(\mathbb{S}\left[{\mathcal{J}}_{\mathbb{H}}\right]\right)}^{\mathrm{T}}\mathbb{S}\left[{\mathcal{A}}_{{\psi }_{k-1}}\right]\mathbb{S}\left[{\mathcal{A}}_{{\psi }_k}{X}_{k-2}(·)\right]={\left(\mathbb{S}\left[{\mathcal{J}}_{\mathbb{H}}\right]\right)}^{\mathrm{T}}\mathbb{S}\left[{\mathcal{A}}_{{\psi }_{k-1}}\right]\mathbb{S}\left[{\mathcal{A}}_{{\psi }_k}\right]\mathbb{S}\left[X_{k-2}(·)\right],\hfill \end{array}$$

i.e.,

$${\mathcal{J}}_{\mathbb{H}}{\mathcal{A}}_{{\psi }_{k-1}}{\mathcal{A}}_{{\psi }_k}{X}_{k-2}(·)={\mathcal{V}}_0^{\mathrm{T}}{\mathrm{\Psi }}_{k-1}{\mathrm{\Psi }}_k{\mathcal{X}}_{k-2},$$

and

$${}^{\mathrm{I}}{\mathcal{J}}_{\mathbb{H}}^{W(j_1\dots j_k)}{\mathbb{k}}_{\psi }(·)=-\frac{1}{2} {\delta }_{j_{k-1}{j}_k}{\mathcal{V}}_0^{\mathrm{T}}{\mathrm{\Psi }}_{k-1}{\mathrm{\Psi }}_k{\mathcal{X}}_{k-2}+{\mathcal{V}}_{j_k}^{\mathrm{T}}{\mathrm{\Psi }}_k{\mathcal{X}}_{k-1}.$$

Thus, Relations (34) are proved. □

Theorem 2 allows one to obtain the spectral representation of iterated Itô stochastic integrals from Expansion (2) (in this case, ${\mathrm{\Psi }}_1=\dots ={\mathrm{\Psi }}_k=E$):

$$\begin{array}{cc}\hfill {\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_1}^{\mathrm{T}}{\mathcal{V}}_0,\hspace{1em}{}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_0^{\mathrm{T}}{\mathcal{V}}_0,\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_1}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_0,\hspace{1em}{}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)={\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1},\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_3}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_{j_3}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_0-\frac{{\delta }_{j_2{j}_3}}{2}{\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1},\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1{j}_2\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_1}\right)P^{-1}{\mathcal{V}}_0-\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_0,\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0j_2\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_0\right)P^{-1}{\mathcal{V}}_{j_1},\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_{2}0\right)}\mathbb{k}(·)& ={\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_0,\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3{j}_4\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_4}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_3}\right)P^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_{j_4}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_3}\right)P^{-1}{\mathcal{V}}_0\hfill \\ & -\frac{{\delta }_{j_2{j}_3}}{2}{\mathcal{V}}_{j_4}^{\mathrm{T}}{P}^{-2}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_3{j}_4}}{2}{\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1}+\frac{{\delta }_{j_1{j}_2}{\delta }_{j_3{j}_4}}{4}{\mathcal{V}}_0^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_0,\hfill \end{array}$$

(38)

where $j_1,j_2,j_3,j_4\in \{1,\dots ,s\}$.

If we use the unified Taylor–Itô expansion from [20], then we need the following iterated Itô stochastic integrals and their spectral representation:

$$\begin{array}{cc}\hfill {\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}{\mathbb{k}}_1(·)& ={\mathcal{V}}_{j_1}^{\mathrm{T}}F,\hspace{1em}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\left(h-{\mathbb{k}}_1(·)\right)={\mathcal{V}}_{j_1}^{\mathrm{T}}(h{\mathcal{V}}_0-F),\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}{\mathbb{k}}_{10}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}A{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_0^{\mathrm{T}}A{\mathcal{V}}_0,\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\left({\mathbb{k}}_{01}(·)-{\mathbb{k}}_{10}(·)\right)& ={}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}{\mathbb{k}}_{01}(·)-{}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}{\mathbb{k}}_{10}(·)={\mathcal{V}}_{j_2}^{\mathrm{T}}A{P}^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_0^{\mathrm{T}}A{\mathcal{V}}_0\hfill \\ & -{\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}A{\mathcal{V}}_{j_1}+\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_0^{\mathrm{T}}A{\mathcal{V}}_0={\mathcal{V}}_{j_2}^{\mathrm{T}}(A{P}^{-1}-P^{-1}A){\mathcal{V}}_{j_1},\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\left(h\mathbb{k}(·)-{\mathbb{k}}_{01}(·)\right)& =h {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)-{}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}{\mathbb{k}}_{01}(·)=h{\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_0^{\mathrm{T}}{\mathcal{V}}_0\hfill \\ & -{\mathcal{V}}_{j_2}^{\mathrm{T}}A{P}^{-1}{\mathcal{V}}_{j_1}+\frac{{\delta }_{j_1{j}_2}}{2} {\mathcal{V}}_0^{\mathrm{T}}A{\mathcal{V}}_0={\mathcal{V}}_{j_2}^{\mathrm{T}}(hE-A)P^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}}{2} {\mathcal{V}}_0^{\mathrm{T}}(hE-A){\mathcal{V}}_0.\hfill \end{array}$$

On the right-hand sides of the above relations, all of the terms containing spectral characteristics ${\mathcal{V}}_{j_1},\dots ,{\mathcal{V}}_{j_4}$ are iterated Stratonovich stochastic integrals. This follows from Theorem 1, and we can use Relations (30) and (31) for them. So, we obtain the final form

$$\begin{array}{cc}\hfill {\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_1}^{\mathrm{T}}{\mathcal{V}}_0,\hspace{1em}{}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}h}{2},\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_1}^{\mathrm{T}}F,\hspace{1em}{}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0\right)}\mathbb{k}(·)={\mathcal{V}}_{j_1}^{\mathrm{T}}(h{\mathcal{V}}_0-F),\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_3}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_{j_3}^{\mathrm{T}}F-\frac{{\delta }_{j_2{j}_3}}{2}{\mathcal{V}}_{j_1}^{\mathrm{T}}(h{\mathcal{V}}_0-F),\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(0j_1{j}_2\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}A{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}{h}^2}{4},\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_{1}0j_2\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}(A{P}^{-1}-P^{-1}A){\mathcal{V}}_{j_1},\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_{2}0\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_2}^{\mathrm{T}}(hE-A)P^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}{h}^2}{4},\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2{j}_3{j}_4\right)}\mathbb{k}(·)& ={\mathcal{V}}_{j_4}^{\mathrm{T}}{P}^{-1}\left(V{\mathcal{V}}_{j_3}\right)P^{-1}\left(V{\mathcal{V}}_{j_2}\right)P^{-1}{\mathcal{V}}_{j_1}-\frac{{\delta }_{j_1{j}_2}}{2}{\mathcal{V}}_{j_4}^{\mathrm{T}}{P}^{-1}A{\mathcal{V}}_{j_3}\hfill \\ & -\frac{{\delta }_{j_2{j}_3}}{2}{\mathcal{V}}_{j_4}^{\mathrm{T}}(A{P}^{-1}-P^{-1}A){\mathcal{V}}_{j_1}-\frac{{\delta }_{j_3{j}_4}}{2}{\mathcal{V}}_{j_2}^{\mathrm{T}}(hE-A)P^{-1}{\mathcal{V}}_{j_1}+\frac{{\delta }_{j_1{j}_2}{\delta }_{j_3{j}_4}{h}^2}{8}.\hfill \end{array}$$

(39)

Therefore, iterated stochastic integrals needed for the implementation of numerical methods (9)–(12) for solving the stochastic differential equation (1) are expressed using algebraic operations with the column matrices ${\mathcal{V}}_0,{\mathcal{V}}_1,\dots ,{\mathcal{V}}_s,F$, matrices $P^{-1}$ and A, and the three-dimensional matrix V.

The same property holds for iterated Itô stochastic integrals as for iterated Stratonovich stochastic integrals, i.e.,

$${}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}{\mathbb{k}}_{n_1\dots n_k}(·)\stackrel{d}{=} h^{k/2+{\sum }_{l=1}^k({\delta }_{j_{l}0}/2+n_l)}·{}^{\mathrm{I}}{\mathcal{J}}_{[0,1]}^{W(j_1\dots j_k)}{\mathbb{k}}_{n_1\dots n_k}^{\ast }(·),$$

(40)

where all of the notations are introduced as above. Thus, if the spectral characteristics ${\mathcal{V}}_0$, F, $P^{-1}$, A and V, with respect to the basis ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$ of $L_2\left([0,1]\right)$, are known, then we have the base of simulation for all the necessary iterated Itô stochastic integrals.

Obviously, Properties (33) and (40) significantly reduce the computational costs for finding spectral characteristics. In addition, they allow one to determine the terms from Expansions (2) and (14) that should be included in the numerical scheme to guarantee the order of convergence p:

$$\begin{array}{cc}\hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}\mathbb{k}(·), {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}\mathbb{k}(·):& \hspace{1em}\frac{k}{2}+\sum _{l=1}^k\frac{{\delta }_{j_{l}0}}{2}⩽p,\hfill \\ \hfill {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}{\mathbb{k}}_{n_1\dots n_k}(·), {}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W(j_1\dots j_k)}{\mathbb{k}}_{n_1\dots n_k}(·):& \hspace{1em}\frac{k}{2}+\sum _{l=1}^k\left(\frac{{\delta }_{j_{l}0}}{2}+n_l\right)⩽p.\hfill \end{array}$$

Equations (32) and (39) give the exact representation, but, in practice, only the approximate simulation of iterated stochastic integrals is possible, except for some integrals. The approximate simulation requires one to use Equations (32) and (39), but all matrices should be truncated with some chosen order.

In this paper, the accuracy of the mean-square approximation of iterated stochastic integrals is not considered since this requires a great deal of additional theoretical results. However, it is possible to find the exact values or the estimates of approximation errors for all iterated stochastic integrals from Expansions (2) and (14). The results from [20,31] can be applied to this. For example, consider the iterated stochastic integrals of a second multiplicity ${}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)$ and ${}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)$ under conditions $j_1\ne j_2$ and $j_1,j_2\ne 0$. The exact representation of them is written as

$${}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)={}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)={\mathcal{V}}_{j_2}^{\mathrm{T}}{P}^{-1}{\mathcal{V}}_{j_1}=\sum _{i_1,i_2=0}^{\infty }{\left({\mathcal{V}}_{j_2}\right)}_{i_1}{P}_{i_1{i}_2}^{-1}{\left({\mathcal{V}}_{j_1}\right)}_{i_2}.$$

Consequently, the approximate representation has the form

$${}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)={}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)\approx {}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\tilde{\mathbb{k}}(·)={}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\tilde{\mathbb{k}}(·)=\sum _{i_1,i_2=0}^{L-1}{\left({\mathcal{V}}_{j_2}\right)}_{i_1}{P}_{i_1{i}_2}^{-1}{\left({\mathcal{V}}_{j_1}\right)}_{i_2},$$

where L is the truncation order and

$$\begin{array}{cc}\hfill \mathbb{k}(t_1,t_2)& =\sum _{i_1,i_2=0}^{\infty }{P}_{i_1{i}_2}^{-1}q(i_1,t_1)q(i_2,t_2)=1(t_2-t_1),\hfill \\ \hfill \tilde{\mathbb{k}}(t_1,t_2)& =\sum _{i_1,i_2=0}^{L-1}{P}_{i_1{i}_2}^{-1}q(i_1,t_1)q(i_2,t_2)\approx 1(t_2-t_1).\hfill \end{array}$$

Then, we have

$$\begin{array}{c}\mathrm{E}{\left({}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)-{}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\tilde{\mathbb{k}}(·)\right)}^2=\mathrm{E}{\left({}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)-{}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\tilde{\mathbb{k}}(·)\right)}^2\hfill \\ \hspace{1em}=\parallel \mathbb{k}(·)-\tilde{\mathbb{k}}{(·)\parallel }_{L_2\left({\mathbb{H}}^2\right)}^2={\parallel \mathbb{k}(·)\parallel }_{L_2\left({\mathbb{H}}^2\right)}^2-\sum _{i_1,i_2=0}^{L-1}{\left(P_{i_1{i}_2}^{-1}\right)}^2,\hspace{1em}{\parallel \mathbb{k}(·)\parallel }_{L_2\left({\mathbb{H}}^2\right)}^2=\frac{h^2}{2},\hfill \end{array}$$

where $L_2\left({\mathbb{H}}^2\right)$ is the space of square-integrable functions $x(·):{\mathbb{H}}^2\to \mathbb{R}$ with the standard norm ${\parallel ·\parallel }_{L_2\left({\mathbb{H}}^2\right)}$. But this example is one of the simplest since the multiplicity $k=2$ is not high and the iterated stochastic integrals ${}^{\mathrm{I}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)$ and ${}^{\mathrm{S}}{\mathcal{J}}_{[\vartheta ,\vartheta +h]}^{W\left(j_1{j}_2\right)}\mathbb{k}(·)$ coincide with probability 1.

In [31], Legendre polynomials, cosines, Walsh and Haar functions, and trigonometric functions are tested as the basis ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$, and cosines (the Fourier series for an even function is based on this basis) provide the best approximation. The use of Walsh and Haar functions is similar to numerical integration [40].

6. Numerical Experiments

Here, we use the obtained spectral representations for the simulation of iterated Itô and Stratonovich stochastic integrals of the multiplicity $k=2,3,4$ under condition $\mathbb{H}=[0,1]$.

As the basis ${\left\{q(i,·)\right\}}_{i=0}^{\infty }$ of $L_2\left([0,1]\right)$, we choose cosines

$$q(i,t)=\left\{\begin{array}{cc}1\hfill & \mathrm{for} i=0\hfill \\ \sqrt{2} cosi\pi t\hfill & \mathrm{for} i>0,\hfill \end{array}\right.$$

and, for this basis, we have the following spectral characteristics of functions, operators, and functionals [30]:

(1) The spectral characteristic ${\mathcal{V}}_0$ of the function $\mathbb{k}\left(t\right)\equiv 1$:

$${\mathcal{V}}_0=\mathbb{S}\left[\mathbb{k}(·)\right],\hspace{1em}{\left({\mathcal{V}}_0\right)}_i=\left\{\begin{array}{cc}1\hfill & \mathrm{for} i=0\hfill \\ 0\hfill & \mathrm{for} i>0;\hfill \end{array}\right.$$

(2) The spectral characteristic F of the function ${\mathbb{k}}_1\left(t\right)=t$:

$$F=\mathbb{S}\left[{\mathbb{k}}_1(·)\right],\hspace{1em}{F}_i=\left\{\begin{array}{cc}1/2& \mathrm{for} i=0\hfill \\ \sqrt{2}[{(-1)}^i-1]/i^2{\pi }^2& \mathrm{for} i>0;\hfill \end{array}\right.$$

(3) The spectral characteristic $P^{-1}$ of the integration operator ${\mathcal{D}}^{-1}$:

$$P^{-1}=\mathbb{S}\left[{\mathcal{D}}^{-1}\right],\hspace{1em}{P}_{i_1{i}_2}^{-1}=\left\{\begin{array}{cc}1/2& \mathrm{for} i_1=i_2=0\hfill \\ \sqrt{2}(1-{(-1)}^{i_2})/i_2^2{\pi }^2& \mathrm{for} i_1=0,i_2>0\hfill \\ -\sqrt{2}(1-{(-1)}^{i_1})/i_1^2{\pi }^2& \mathrm{for} i_1>0,i_2=0\hfill \\ 0& \mathrm{for} i_1=i_2>0\hfill \\ 2(1-{(-1)}^{i_2-i_1})/k(i_1+i_2){\pi }^2& \mathrm{for} 0<i_1<i_2\hfill \\ -2(1-{(-1)}^{i_1-i_2})/k(i_1+i_2){\pi }^2& \mathrm{for} 0<i_2<i_1;\hfill \end{array}\right.$$

(4) The spectral characteristic A of the multiplication operator ${\mathcal{A}}_{{\mathbb{k}}_1}$:

$$A=\mathbb{S}\left[{\mathcal{A}}_{{\mathbb{k}}_1}\right],\hspace{1em}{A}_{i_1{i}_2}=\left\{\begin{array}{cc}1/2& \mathrm{for} i_1=i_2\hfill \\ \sqrt{2}[{(-1)}^{i_2}-1]/i_2^2{\pi }^2& \mathrm{for} i_1=0,i_2>0\hfill \\ \sqrt{2}[{(-1)}^{i_1}-1]/i_1^2{\pi }^2& \mathrm{for} i_1>0,i_2=0\hfill \\ 2(i_1^2+i_2^2)[{(-1)}^{i_2-i_1}-1]/{(i_2-i_1)}^2{(i_1+i_2)}^2{\pi }^2& \mathrm{for} 0<i_1<i_2\hfill \\ 2(i_1^2+i_2^2)[{(-1)}^{i_1-i_2}-1]/{(i_1-i_2)}^2{(i_1+i_2)}^2{\pi }^2& \mathrm{for} 0<i_2<i_1;\hfill \end{array}\right.$$

(5) The spectral characteristic V of the bilinear multiplication operator $\mathcal{M}$:

$$V=\mathbb{S}\left[\mathcal{M}\right],\hspace{1em}{V}_{i_1{i}_2{i}_3}=\left\{\begin{array}{cc}1/\sqrt{1+(i_1>0)(i_2>0)(i_3>0)}\hfill & \mathrm{for} |i_1-i_2|=i_3 \mathrm{or} i_1+i_2=i_3\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

where $(i>0)=0$ if $i=0$ and $(i>0)=1$ if $i>0$;

(6) The spectral characteristics ${\mathcal{V}}_1,\dots ,{\mathcal{V}}_s$ of Gaussian white noises $V_1(·),\dots ,V_s(·)$ consist of independent random variables having standard normal distribution.

The above relations for spectral characteristics provide an approximate simulation of iterated Itô and Stratonovich stochastic integrals of arbitrary multiplicity from Expansions (2) and (14), or from the corresponding unified expansions.

Further, we estimate the first two moments of the iterated stochastic integrals ${}^{\mathrm{I}}{\mathcal{J}}_{[0,1]}^{W(j_1\dots j_k)}\mathbb{k}(·)$ and ${}^{\mathrm{S}}{\mathcal{J}}_{[0,1]}^{W(j_1\dots j_k)}\mathbb{k}(·)$ using the corresponding spectral representations for simulation. The truncation orders of spectral characteristics are $L=4,8,\dots ,64$, i.e., all matrices have a size L for any dimension.

For each truncation order L, we obtain the realizations of both iterated Itô stochastic integrals ${\left\{I_m\right\}}_{m=1}^M$ and iterated Stratonovich stochastic integrals ${\left\{S_m\right\}}_{m=1}^M$, where $M={10}^6$ is the sample size. The expectation and the second moment may be estimated by formulas

$$\begin{array}{cc}\mathrm{E}{}^{\mathrm{I}}{\mathcal{J}}_{[0,1]}^{W(j_1\dots j_k)}\mathbb{k}(·)\approx \frac{1}{M}\sum _{m=1}^M{I}_m,\hfill & \hfill \mathrm{E}{\left({}^{\mathrm{I}}{\mathcal{J}}_{[0,1]}^{W(j_1\dots j_k)}\mathbb{k}(·)\right)}^2\approx \frac{1}{M}\sum _{m=1}^M{I}_m^2,\\ \mathrm{E}{}^{\mathrm{S}}{\mathcal{J}}_{[0,1]}^{W(j_1\dots j_k)}\mathbb{k}(·)\approx \frac{1}{M}\sum _{m=1}^M{S}_m,\hfill & \hfill \mathrm{E}{\left({}^{\mathrm{S}}{\mathcal{J}}_{[0,1]}^{W(j_1\dots j_k)}\mathbb{k}(·)\right)}^2\approx \frac{1}{M}\sum _{m=1}^M{S}_m^2.\end{array}$$

The estimates of the expectation and the second moment are given in Table 1 and Table 2 (in these tables, the symbol ∗ means “pairwise distinct”), respectively. The results for the iterated stochastic integrals of a fourth multiplicity are partially presented since the total number of different cases is too large, but the remaining integrals are similar in their properties.

Table 1. The estimates of the expectation of iterated Itô (I) and Stratonovich (S) stochastic integrals.

Type	Indexes	$\mathit{L}=4$	$\mathit{L}=8$	$\mathit{L}=16$	$\mathit{L}=32$	$\mathit{L}=64$	Exact
$k = 2$
I, S	$j_1\ne j_2$	0.000060	0.000876	−0.001349	0.000281	0.000434	0
I	$j_1=j_2$	−0.000758	−0.000616	−0.000029	−0.000187	0.000034	0
S	$j_1=j_2$	0.499242	0.499384	0.499971	0.499813	0.500034	1/2
I, S	$j_1=0,j_2\ne 0$	−0.000535	0.000127	−0.000244	0.000821	0.001160	0
I, S	$j_1\ne 0,j_2=0$	−0.001284	0.000640	−0.000680	0.000346	−0.000455	0
$k = 3$
I, S	$j_1,j_2,j_3$∗	0.000439	0.000092	−0.000645	−0.000003	−0.000473	0
I	$j_1=j_2\ne j_3$	0.000190	0.000473	0.000057	−0.000018	0.000113	0
S	$j_1=j_2\ne j_3$	0.000531	0.000386	0.000308	0.000136	0.000490	0
I, S	$j_1=j_3\ne j_2$	0.000228	−0.000588	−0.000414	−0.000665	−0.000616	0
I	$j_1\ne j_2=j_3$	0.000429	0.000257	−0.000346	−0.000567	0.000623	0
S	$j_1\ne j_2=j_3$	0.000416	0.001048	−0.000406	−0.000416	−0.000193	0
I	$j_1=j_2=j_3$	−0.000450	−0.000089	−0.000064	−0.000037	0.000101	0
S	$j_1=j_2=j_3$	−0.000017	0.001379	−0.000369	−0.000111	0.000431	0
I, S	$j_1=0$; $0,j_2,j_3$∗	−0.000318	−0.000450	−0.000073	−0.000035	0.000063	0
I	$j_1=0$; $0\ne j_2=j_3$	−0.000023	0.000024	−0.000307	0.000446	−0.000033	0
S	$j_1=0$; $0\ne j_2=j_3$	0.249977	0.250024	0.249693	0.250446	0.249967	1/4
I, S	$j_2=0$; $0,j_1,j_3$∗	−0.000471	−0.000354	−0.000068	0.000305	0.000429	0
I, S	$j_2=0$; $0\ne j_1=j_3$	0.034169	0.014876	0.007123	0.003134	0.001025	0
I, S	$j_3=0$; $0,j_1,j_2$∗	−0.000146	−0.000469	0.000233	0.000131	0.000232	0
I	$j_3=0$; $0\ne j_1=j_2$	−0.000070	−0.000539	0.000156	−0.000139	−0.000417	0
S	$j_3=0$; $0\ne j_1=j_2$	0.249930	0.249461	0.250156	0.249861	0.249583	1/4
$k = 4$
I, S	$j_1,j_2,j_3,j_4$∗	0.000206	0.000197	0.000006	0.000222	0.000064	0
I	$j_1=j_2$; $j_2,j_3,j_4$∗	0.000214	0.000233	−0.000062	0.000181	−0.000006	0
S	$j_1=j_2$; $j_2,j_3,j_4$∗	0.000244	0.000266	−0.000269	0.000202	0.000072	0
I	$j_2=j_3$; $j_1,j_3,j_4$∗	0.000232	0.000161	0.000048	0.000002	0.000069	0
S	$j_2=j_3$; $j_1,j_3,j_4$∗	0.000025	0.000069	0.000114	0.000152	−0.000144	0
I	$j_3=j_4$; $j_1,j_2,j_3$∗	0.000027	−0.000215	−0.000132	0.000028	0.000647	0
S	$j_3=j_4$; $j_1,j_2,j_3$∗	−0.000130	−0.000358	−0.000220	−0.000146	0.000642	0
I	$j_1=j_2\ne j_3=j_4$	0.000302	−0.000172	0.000056	−0.000297	−0.000267	0
S	$j_1=j_2\ne j_3=j_4$	0.125107	0.124648	0.125088	0.124751	0.124513	1/8
I	$j_1=j_2=j_3\ne j_4$	0.000216	0.000226	0.000085	0.000247	0.000188	0
S	$j_1=j_2=j_3\ne j_4$	0.000181	−0.000075	0.000050	0.000609	0.000295	0
I	$j_1\ne j_2=j_3=j_4$	0.000244	−0.000224	0.000035	0.000141	−0.000388	0
S	$j_1\ne j_2=j_3=j_4$	0.000196	−0.000040	0.000205	0.000521	−0.000094	0
I	$j_1=j_2=j_3=j_4$	0.017100	0.007091	0.003595	0.002127	0.000586	0
S	$j_1=j_2=j_3=j_4$	0.158683	0.139131	0.132259	0.128635	0.126425	1/8

Table 2. The estimates of the second moment of iterated Itô (I) and Stratonovich (S) stochastic integrals.

Type	Indexes	$\mathit{L}=4$	$\mathit{L}=8$	$\mathit{L}=16$	$\mathit{L}=32$	$\mathit{L}=64$	Exact
$k = 2$
I, S	$j_1\ne j_2$	0.465907	0.483632	0.492898	0.495102	0.498588	1/2
I	$j_1=j_2$	0.499816	0.498611	0.498799	0.499023	0.500765	1/2
S	$j_1=j_2$	0.749058	0.747995	0.748770	0.748836	0.750799	3/4
I, S	$j_1=0,j_2\ne 0$	0.333535	0.332759	0.334333	0.333908	0.333459	1/3
I, S	$j_1\ne 0,j_2=0$	0.332678	0.333781	0.333898	0.333465	0.333364	1/3
$k = 3$
I, S	$j_1,j_2,j_3$∗	0.132299	0.151835	0.160826	0.163014	0.163898	1/6
I	$j_1=j_2\ne j_3$	0.149313	0.159617	0.164400	0.165510	0.164671	1/6
S	$j_1=j_2\ne j_3$	0.231989	0.243090	0.248019	0.248301	0.249108	1/4
I, S	$j_1=j_3\ne j_2$	0.134000	0.151443	0.158948	0.163769	0.164130	1/6
I	$j_1\ne j_2=j_3$	0.148604	0.159427	0.163231	0.165468	0.166373	1/6
S	$j_1\ne j_2=j_3$	0.231337	0.242854	0.246821	0.247515	0.248243	1/4
I	$j_1=j_2=j_3$	0.174227	0.168281	0.166097	0.168727	0.166087	1/6
S	$j_1=j_2=j_3$	0.451087	0.432105	0.422265	0.423596	0.421706	5/12
I, S	$j_1=0$; $0,j_2,j_3$∗	0.071585	0.078284	0.081038	0.082065	0.082623	1/12
I	$j_1=0$; $0\ne j_2=j_3$	0.084121	0.083223	0.082760	0.083458	0.083549	1/12
S	$j_1=0$; $0\ne j_2=j_3$	0.146609	0.145736	0.145106	0.146181	0.146032	7/48
I, S	$j_2=0$; $0,j_1,j_3$∗	0.082875	0.083242	0.083606	0.083463	0.083384	1/12
I, S	$j_2=0$; $0\ne j_1=j_3$	0.083459	0.083667	0.083266	0.082823	0.082827	1/12
I, S	$j_3=0$; $0,j_1,j_2$∗	0.071705	0.078179	0.080789	0.082156	0.082750	1/12
I	$j_3=0$; $0\ne j_1=j_2$	0.083897	0.082929	0.083296	0.083296	0.082747	1/12
S	$j_3=0$; $0\ne j_1=j_2$	0.146362	0.145159	0.145874	0.145727	0.145039	7/48
$k = 4$
I, S	$j_1,j_2,j_3,j_4$∗	0.026284	0.034340	0.038080	0.039713	0.040704	1/24
I	$j_1=j_2$; $j_2,j_3,j_4$∗	0.032121	0.036981	0.039293	0.040423	0.041142	1/24
S	$j_1=j_2$; $j_2,j_3,j_4$∗	0.050212	0.056589	0.059438	0.061003	0.061857	1/16
I	$j_2=j_3$; $j_1,j_3,j_4$∗	0.031948	0.036993	0.039712	0.040393	0.040891	1/24
S	$j_2=j_3$; $j_1,j_3,j_4$∗	0.052415	0.057739	0.060423	0.061155	0.061768	1/16
I	$j_3=j_4$; $j_1,j_2,j_3$∗	0.031347	0.037081	0.038916	0.040413	0.041087	1/24
S	$j_3=j_4$; $j_1,j_2,j_3$∗	0.048897	0.056715	0.058994	0.061011	0.061813	1/16
I	$j_1=j_2\ne j_3=j_4$	0.038102	0.039869	0.040555	0.040781	0.040494	1/24
S	$j_1=j_2\ne j_3=j_4$	0.096154	0.096997	0.098125	0.097580	0.096876	19/192
I	$j_1=j_2=j_3\ne j_4$	0.039374	0.039681	0.041205	0.040540	0.040751	1/24
S	$j_1=j_2=j_3\ne j_4$	0.109809	0.105472	0.105562	0.102574	0.102797	5/48
I	$j_1\ne j_2=j_3=j_4$	0.040374	0.039917	0.040760	0.040617	0.040515	1/24
S	$j_1\ne j_2=j_3=j_4$	0.111217	0.105296	0.104803	0.103760	0.102975	5/48
I	$j_1=j_2=j_3=j_4$	0.049875	0.043431	0.042314	0.043318	0.040078	1/24
S	$j_1=j_2=j_3=j_4$	0.235501	0.201176	0.192693	0.191046	0.180914	35/192

For the iterated stochastic integrals, there are no analytical expressions to describe their distribution laws, except for some integrals. However, it is quite simple to find the expectation and the second moment, so we can compare the estimates of the expectation and the second moment with the known exact values. All the results are consistent with the exact values, which are given in the last column of Table 1 and Table 2.

In general, the estimation results contain errors of two types. The first error is the methodological error of the spectral method, and it is associated with the truncation of all matrices with the order L for any dimension. The second error is the Monte Carlo error, which depends on the sample size M only.

For example, the first error dominates for rows $k=2$ ($I,S$) $j_1\ne j_2$, $k=3$ ($I,S$) $j_1,j_2,j_3\ast$, and $k=4$ ($I,S$) $j_1,j_2,j_3,j_4\ast$ in Table 2; however, the convergence to the exact values is obvious. There are many such rows in Table 1 and Table 2. But for some rows, there exist only a second error, for example, this is true for rows $k=2$ ($I,S$) $j_1\ne j_2$, $k=3$ ($I,S$) $j_1,j_2,j_3\ast$, and $k=4$ ($I,S$) $j_1,j_2,j_3,j_4\ast$ in Table 1. This is also true for some other rows of Table 1 and Table 2; however, there, a more detailed analysis is required.

7. Discussion

This paper considers numerical methods for solving stochastic differential equations with the order of mean-square and strong convergence $p=0.5,1.0,1.5,2.0$; however, using the obtained spectral representations, methods with the order of convergence $2.5$ and more can also be considered. There are no restrictions on both the multiplicity of the iterated stochastic integrals and the order of convergence of numerical methods.

For the Euler–Maruyama method, the spectral method is certainly not required (the strong 0.5 order method). For Milstein-type methods (the strong 1.0 order methods), the described approach gives the matrix notation and the possibility of using different bases only. The advantage of the spectral representations appears for methods with the order of convergence 1.5 and more. The approximate simulation of iterated Itô and Stratonovich stochastic integrals of arbitrary multiplicity uses matrix algebra techniques, and it therefore has a very simple implementation. In addition, there is no need to expand functions $\mathbb{k}(·)$ and ${\mathbb{k}}_{n_1\dots n_k}(·)$ with different k into generalized multiple Fourier series, but it is sufficient to use the spectral characteristics ${\mathcal{V}}_0$, F, $P^{-1}$, A, and V.

Through using the proposed approach, we can obtain a similar spectral representation of the iterated stochastic $\theta$-integrals. From this more general representation, Theorems 1 and 2 are obtained as two particular cases: $\theta =0$ for iterated Itô stochastic integrals, and $\theta =1/2$ for iterated Stratonovich stochastic integrals.

8. Conclusions

The main result is the spectral representations of iterated Itô and Stratonovich stochastic integrals for the implementation of numerical methods for solving Itô and Stratonovich stochastic differential equations with high orders of mean-square and strong convergence. Their advantage is provided by simple final forms, namely each iterated stochastic integral is represented by matrix algebra techniques with known matrices. For the approximate simulations of iterated Itô and Stratonovich stochastic integrals, the standard mathematics software are sufficient, see—for example—the software based on the BLAS (Basic Linear Algebra Subprograms) specification or the software for mathematical calculations, such as Mathcad, MATLAB, etc.