Berry–Esseen Bounds of the Quasi Maximum Likelihood Estimators for the Discretely Observed Diffusions

Abstract

For stationary ergodic diffusions satisfying nonlinear homogeneous Itô stochastic differential equations, this paper obtains the Berry–Esseen bounds on the rates of convergence to normality of the distributions of the quasi maximum likelihood estimators based on stochastic Taylor approximation, under some regularity conditions, when the diffusion is observed at equally spaced dense time points over a long time interval, the high-frequency regime. It shows that the higher-order stochastic Taylor approximation-based estimators perform better than the basic Euler approximation in the sense of having smaller asymptotic variance.

1. Introduction and Preliminaries

Parameter estimation in diffusion processes based on discrete observations is the recent trend of investigation in financial econometrics and mathematical biology since the data available in finance and biology are high-frequency discrete, though the model is continuous. For a treatise on this subject, see Bishwal (2008, 2021) [1,2].

Consider the Itô stochastic differential equation

$$\begin{array}{ccc}d{X}_t\hfill & =& {\displaystyle f(\theta ,X_t)dt+d{W}_t,t\ge 0}\hfill \\ X_0\hfill & =& {\displaystyle X^0}\hfill \end{array}$$

(1)

where $\{W_t,t\ge 0\}$ is a one-dimensional standard Wiener process, $\theta \in \Theta$, $\Theta$ is a compact subset of ℝ, f is a known real valued function defined on $\Theta \times ℝ$, the unknown parameter $\theta$ is to be estimated on the basis of observation of the process $\{X_t,t\ge 0\}$. Let ${\theta }_0$ be the true value of the parameter that is in the interior of $\Theta$. We assume that the process $\{X_t,t\ge 0\}$ is observed at $0=t_0<t_1<\dots <t_n=T$ with $\Delta t_i:=t_i-t_{i-1}=\frac{T}{n}=h$ and $T=d{n}^{1/2}$ for some fixed real number $d>0$. We estimate $\theta$ from the observations $\{X_{t_0},X_{t_1},\dots ,X_{t_n}\}$.

The conditional least squares estimator (CLSE) of $\theta$ is defined as

$$\begin{array}{c}{\theta }_{n,T}:=arg{\displaystyle \underset{\theta \in \Theta }{min}}{Q}_{n,T}\left(\theta \right)\\ \mathrm{where}{Q}_{n,T}\left(\theta \right)={\displaystyle \sum _{i=1}^n}\frac{{\left[X_{t_i}-X_{t_{i-1}}-f(\theta ,X_{t_{i-1}})h\right]}^2}{\Delta t_i}.\end{array}$$

(2)

This estimator was first studied by Dorogovcev (1976) [3], who obtained its weak consistency under some regularity conditions as $T\to \infty$ and $\frac{T}{n}\to 0$. Kasonga (1988) [4] obtained the strong consistency of the CLSE under some regularity conditions as $n\to \infty$ assuming that $T=d{n}^{1/2}$ for some fixed real number $d>0$. Prakasa Rao (1983) [5] obtained asymptotic normality of the CLSE as $T\to \infty$ and $\frac{T}{n^{1/2}}\to 0$.

Florens-Zmirou (1989) [6] studied the minimum contrast estimator, based on a Euler–Maruyama-type first-order approximate discrete time scheme of the SDE (1), which is given by

$$Z_{t_i}-Z_{t_{i-1}}=f(\theta ,Z_{t_{i-1}})(t_i-t_{i-1})+W_{t_i}-W_{t_{i-1}},i\ge 1,Z_0=X^0.$$

(3)

The log-likelihood function of $\{Z_{t_i},0\le i\le n\}$ is given by

$$L_{n,T}=C\sum _{i=1}^n\frac{{\left[Z_{t_i}-Z_{t_{i-1}}-f(\theta ,Z_{t_{i-1}})h\right]}^2}{\Delta t_i}.$$

(4)

where C is a constant independent of $\theta$. A contrast for the estimation of $\theta$ is derived from the above log-likelihood by substituting $\{Z_{t_i},0\le i\le n\}$ with $\{X_{t_i},0\le i\le n\}$. The resulting contrast is

$$H_{n,T}=C\sum _{i=1}^n\frac{{\left[X_{t_i}-X_{t_{i-1}}-f(\theta ,X_{t_{i-1}})h\right]}^2}{\Delta t_i}$$

(5)

and the resulting minimum contrast estimator, called the Euler–Maruyama estimator, is given by

$${\check{\theta }}_{n,T}:=arg\underset{\theta \in \Theta }{min}{H}_{n,T}\left(\theta \right)$$

Florens-Zmirou (1989) [6] showed the $L_2$-consistency of the estimator as $T\to \infty$ and $\frac{T}{n}\to 0$ and asymptotic normality as $T\to \infty$ and $\frac{T}{n^{2/3}}\to 0$.

Notice that the contrast $H_{n,T}$ would be the log-likelihood of $(X_{t_i},0\le i\le n)$ if the transition probability was $\mathcal{N}\left(f\right(\theta ,x)h,h))$. This led Kessler (1997) [7] to consider Gaussian approximation of the transition density. The most natural one is achieved through choosing its mean and variance to be the mean and variance of the transition density. Thus, the transition density is approximated by $\mathcal{N}(E\left(X_{t_i}\right|X_{t_{i-1}}),h))$, which produces the contrast

$$K_{n,T}=C\sum _{i=1}^n\frac{{\left[X_{t_i}-E\left(X_{t_i}\right|X_{t_{i-1}})\right]}^2}{\Delta t_i}.$$

(6)

Since the transition density is unknown, in general, there is no closed-form expression for $E\left(X_{t_i}\right|X_{t_{i-1}})$. Using the stochastic Taylor formula obtained in Florens-Zmirou (1989) [6], he obtained a closed-form expression of $E\left(X_{t_i}\right|X_{t_{i-1}}).$ The contrast $H_{n,T}$ is an example of such an approximation when $E\left(X_{t_i}\right|X_{t_{i-1}})\approx X_{t_{i-1}}+hf(\theta ,X_{t_{i-1}})$.

The resulting minimum contrast estimator, which is also the quasi-maximum likelihood estimator (QMLE), is given by

$${\theta }_{n,T}:=arg\underset{\theta \in \Theta }{min}{K}_{n,T}\left(\theta \right)$$

Kessler (1997) [7] showed the $L_2$-consistency of the estimator as $T\to \infty$ and $\frac{T}{n}\to 0$ and asymptotic normality as $T\to \infty$ and $\frac{T}{n^{(p-1)/p}}\to 0$ for an arbitrary integer p.

Denote

$$\mu (\theta ,X_{t_{i-1}}):=E\left(X_{t_i}\right|X_{t_{i-1}}),\mu (\theta ,x):=E\left(X_{t_i}\right|X_{t_{i-1}}=x)$$

(7)

which is the mean function of the transition probability distribution. Hence, the contrast is given by

$$K_{n,T}=C\sum _{i=1}^n\frac{{\left[X_{t_i}-\mu (\theta ,X_{t_{i-1}}))\right]}^2}{\Delta t_i}.$$

(8)

If continuous observation of $\left\{X_t\right\}$ on the interval $[0,T]$ were available, then the likelihood function of $\theta$ would be

$$L_T\left(\theta \right)=exp\left\{{\int }_0^{T}f(\theta ,X_t)d{X}_t-\frac{1}{2}{\int }_0^T{f}^2(\theta ,X_t)dt\right\}$$

(9)

(see Liptser and Shiryayev (1977) [8]). Since we have discrete data, we have to approximate the likelihood to obtain the MLE. Taking Itô-type approximation of the stochastic integral and rectangle rule approximation of the ordinary integral in (9), we obtain the approximate likelihood function

$${\widehat{L}}_{n,T}\left(\theta \right):=exp\left\{\sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})(X_{t_i}-X_{t_{i-1}})-\frac{h}{2}\sum _{i=1}^n{f}^2(\theta ,X_{t_{i-1}})\right\}.$$

(10)

The Itô approximate maximum likelihood estimate (IAMLE) based on ${\widehat{L}}_{n,T}$ is defined as

$${\widehat{\theta }}_{n,T}:=arg\underset{\theta \in \Theta }{max}{\widehat{L}}_{n,T}\left(\theta \right).$$

Weak consistency and asymptotic normality of this estimator were obtained by Yoshida (1992) [9] as $T\to \infty$ and $\frac{T}{n}\to 0$.

Note that the CLSE, the Euler–Maruyama estimator and the IAMLE are the same estimator (see Shoji (1997) [10]). For the Ornstein–Uhlenbeck process, Bishwal and Bose (2001) [11] studied the rates of weak convergence of approximate maximum likelihood estimators, which are of conditional least squares type. For the Ornstein–Uhlenbeck process, Bishwal (2010) [12] studied the uniform rate of weak convergence for the minimum contrast estimator, which has a close connection to the Stratonovich–Milstein scheme. Bishwal (2009) [13] studied Berry–Esseen inequalities for conditional least squares estimator in discretely observed nonlinear diffusions. Bishwal (2009) [14] studied the Stratonovich-based approximate M-estimator of discretely sampled nonlinear diffusions. Bishwal (2011) [15] studied Milstein approximation of the posterior density of diffusions. Bishwal (2010) [16] studied conditional least squares estimation in nonlinear diffusion processes based on Poisson sampling. Bishwal (2011) [17] obtained some new estimators of integrated volatility using the stochastic Taylor-type schemes, which could be useful for option pricing in stochastic volatility models; see also Bishwal (2021) [2].

Prime denotes the derivative with respect to $\theta$, dot denotes the derivative with respect to x and ⋁ denotes the max symbol throughout the paper. In order to obtain a better estimator in terms of lowering variance in Monte Carlo simulation, which may have a faster rate of convergence, first, we use the algorithm proposed in Bishwal (2008) [1]. Note that the Itô integral and the Fisk–Stratonovich (FS, henceforth; Fisk, while introducing the concept of quasimartingale, had the trapezoidal approximation and Stratonovich had the midpoint approximation, converging to the same mean square limit) integral are connected by

$${\int }_0^{T}f(\theta ,X_t)d{X}_t={\int }_0^{T}f(\theta ,X_t) \mathrm{o} d{X}_t-\frac{1}{2}{\int }_0^T\dot{f}(\theta ,X_t)dt,$$

(11)

where o is the Itô’s circle for the FS integral. We transform the Itô integral (the limit of the rectangular approximation to preserve the martingale property) in (9) to the FS integral and apply FS-type trapezoidal approximation of the stochastic integral and rectangular rule-type approximation of the Lebesgue integrals and obtain the approximate likelihood

$$\begin{array}{c}\hfill {\tilde{L}}_{n,T}\left(\theta \right):=exp\{\frac{1}{2}\sum _{i=1}^n[f(\theta ,X_{t_{i-1}})+f(\theta ,X_{t_i})](X_{t_i}-X_{t_{i-1}})\\ \hfill -\frac{h}{2}\sum _{i=1}^n\dot{f}(\theta ,X_{t_{i-1}})-\frac{h}{2}{\displaystyle \sum _{i=1}^n}{f}^2(\theta ,X_{t_{i-1}})\}\end{array}$$

(12)

The Fisk–Stratonovich approximate maximum likelihood estimator (FSAMLE) based on ${\tilde{L}}_{n,T}$ is defined as

$${\tilde{\theta }}_{n,T}:=arg\underset{\theta \in \Theta }{max}{\tilde{L}}_{n,T}\left(\theta \right).$$

Weak consistency as $T\to \infty$ and $\frac{T}{n}\to 0$ and asymptotic normality as $T\to \infty$ and $\frac{T}{n^{2/3}}\to 0$ of the FSAMLE were shown in Bishwal (2008) [1]. Berry–Esseen bounds for the IAMLE and the FSAMLE for the Ornstein–Uhlenbeck processes were obtained in Bishwal and Bose (2001) [11].

We shall use the following notations: $\Delta X_i=X_{t_i}-X_{t_{i-1}}$, $\Delta W_i=W_{t_i}-W_{t_{i-1}}$, C is a generic constant independent of $h,n$ and other variables (it may depend on $\theta$). Throughout the paper, $\dot{f}$ denotes the derivative with respect to x and $f^{\prime }$ denotes the derivative with respect to $\theta$ of the function $f(\theta ,x)$. Suppose that ${\theta }_0$ denotes the true value of the parameter and ${\theta }_0\in \Theta$. We assume the following conditions:

Assumption 1.

(A1) ${\displaystyle \left|f\right(\theta ,x\left)\right|\le a\left(\theta \right)(1+|x\left|\right)}$,

         $\left|f\right(\theta ,x)-f(\theta ,y\left)\right|\le a\left(\theta \right)|x-y|$.

(A2) $\left|f\right(\theta ,x)-f(\varphi ,y\left)\right|\le b\left(x\right)|\theta -\varphi |$ for all $\theta ,\varphi \in \Theta ,x,y\in ℝ$

where ${sup}_{\theta \in \Theta }\left|a\left(\theta \right)\right|=a<\infty ,E|b\left(X^0\right){|}^r<\infty$ for any integer r.

(A3) The diffusion process X is stationary and ergodic with invariant measure ν, i.e., for any g with $E\left[g\right(·\left)\right]<\infty ,$

$$\frac{1}{n}\sum _{i=1}^{n}g\left(X_{t_i}\right)\to E_{\nu }\left[g\left(X_0\right)\right]a.s.asT\to \infty andh\to 0.$$

(A4) ${sup}_{t\ge 0}E|X_t{|}^q<\infty$ for all $q\ge 0$.

(A5) $E|f(\theta ,X^0)-f({\theta }_0,X^0){|}^2=0iff\theta ={\theta }_0.$

(A6) f is continuously differentiable function in x up to order p for all θ.

(A7) $f(·,x)$ and all its derivatives are three times continuously differentiable with respect to θ for all $x\in ℝ$. Moreover, these derivatives up to third order with respect to θ are of polynomial growth in x uniformly in θ.

The Fisher information is given by

$$0<I\left(\theta \right):={\int }_{-\infty }^{\infty }{\left(f^{\prime }(\theta ,x)\right)}^{2}d\nu \left(x\right)<\infty$$

and for any $\delta >0$, or any compact $\overline{\Theta }\subset \Theta$,

$$\underset{{\theta }_0\in \overline{\Theta }}{inf}\underset{|\theta -{\theta }_0|>\delta }{sup}{E}_{{\theta }_0}{|f^{\prime }(\theta ,X_0)-f^{\prime }({\theta }_0,X_0)|}^2>0.$$

(A8) The Malliavin covariance of the process is nondegenerate.

The Malliavin covariance matrix of a smooth random variable S is defined as ${\gamma }_T={\int }_0^T{D}_{t}S\left[D_{t}S\right]$${}^{*}dt$, where $D_t$ is the Malliavin derivative. The Malliavin covariance is nondegenerate if $det\left({\gamma }_T\right)$ is almost surely positive and, for any $m\ge 1$, one has $\parallel 1/det\left({\gamma }_T\right){\parallel }_{L^m}<\infty .$ This, associated with the functional $\omega \to X(t,\omega )$, is given by $0<{\sigma }^2\left(t\right)=Y_t^2{\int }_0^t{f}^2(\theta ,X_s)Z_s^{2}ds<\infty$ where $Y_t$ and $Z_t$, respectively, satisfy

$$d{Y}_t=\dot{f}(\theta ,X_t)Y_{t}dt+Y_{t}d{W}_t,Y_0=1,d{Z}_t=-\dot{f}(\theta ,X_t)Z_{t}dt-Z_{t}d{W}_t,Z_0=1.$$

In the case of independent observations, in order to prove the validity of asymptotic expansion, one usually needs a certain regularity condition for the underlying distribution, such as the Cramér condition; see Bhattacharya and Ranga Rao (1976) [18]. This type of condition then ensures the regularity of the distribution and hence the smoothness assumption of the functional under the expectation whose martingale expansion is desired can be removed. This type of condition for dependent observations leads to the regularity of the distribution of a functional with nondegenerate Malliavin covariance, which is known in Malliavin calculus; see Ikeda and Watanabe (1989) [19] and Nualart (1995) [20]. Malliavin covariance is connected to the Hörmander condition, which is a sufficient condition for a second-order differential operator to be hypoelliptic; see Bally (1991) [21]. For operators with analytic coefficients, this condition turns out to be also necessary, but this is not true for general smooth coefficients.

More precisely, let X be a differentiable ℝ-valued Wiener functional defined on a Wiener space. Assume that there exists a functional $\psi$ such that

$$\underset{u\in ℝ}{sup}{\left|u\right|}^{j}E\left[e^{iuX}{X}^k\psi \right]<\infty ,j,k\in Z^{+}.$$

Thus, it is a regularity condition of the characteristic function, which is a consequence of the nondegeneracy of the Malliavin covariance in the case of Wiener functionals. The functional $\psi$, which is a random variable satisfying $0\le \psi \le 1$, is a truncation functional extracting from the Wiener space, the portion on which the distribution is regular. If X is almost regular, one may take $\psi$ nearly equal to one. Uniform degeneracy of the Malliavin covariance of the functional $T^{-1/2}{\int }_0^{T}f({\theta }_0,X_t)d{W}_t$ can be shown under (A8); see Yoshida (1997) [22].

Bishwal (2009) [13] obtained the rate of convergence to normality of the Itô AMLE and the Fisk–Stratonovich AMLE of the order $O\left(T^{-1/2}\bigvee \frac{T^2}{n}\right)$ and $O\left(T^{-1/2}\bigvee \frac{T^3}{n^2}\right)$, respectively, under the regularity conditions given above with $q>16$ for (A4). We obtain the rate of convergence to normality, i.e., Berry–Esseen bound of the order $O\left(T^{-1/2}\bigvee \frac{T^{p+1}}{n^p}\right)$ for the QMLE ${\theta }_{n,T}$ for arbitrary integer p.

We need the following lemma from Michel and Pfanzagl (1971) [23] to prove our main results.

Lemma 1.

Let $\xi ,\zeta$ and η be any three random variables on a probability space $(\Omega ,\mathcal{F},P)$ with $P(\eta >0)=1$. Then, for any $ϵ>0$, we have

$$\left(a\right)\underset{x\in ℝ}{sup}|P\{\xi +\zeta \le x\}-\Phi \left(x\right)|\le \underset{x\in ℝ}{sup}|P\{\xi \le x\}-\Phi \left(x\right)|+P\left(\right|\zeta |>ϵ)+ϵ,$$

$$\left(b\right)\underset{x\in ℝ}{sup}|P\{\frac{\xi }{\eta }\le x\}-\Phi \left(x\right)|\le \underset{x\in ℝ}{sup}|P\{\xi \le x\}-\Phi \left(x\right)|+P\{|\eta -1|>ϵ\}+ϵ.$$

2. Main Results

We start with some preliminary lemmas. Let L denote the generator of the diffusion process, $g\in C^2(ℝ)$

$$Lg\left(x\right):=f(\theta ,x)\dot{g}\left(x\right)+\frac{1}{2}\ddot{g}\left(x\right).$$

The k-th iterate of L is denoted as $L^k$. Its domain is $C^{2k}(ℝ)$. We set $L^0=I$.

Stochastic Taylor formula (Kloeden and Platen (1992) [24]): For a $p+1$ times continuously differentiable function $g:ℝ\to ℝ$, we have for $t\in [0,T]$ and $p=1,2,3,\dots$

$$g\left(X_t\right)=g\left(X_0\right)+\sum _{k=1}^p\frac{t^k}{k!}{L}^{k}g\left(X_0\right)+{\int }_0^t\dots {\int }_0^{s_2}{L}^{p+1}g\left(X_{s_1}\right)d{s}_1\dots d{s}_{p+1}.$$

Lemma 2.

With $f\left(x\right)=x$, the stochastic Taylor expansion of $\mu (\theta ,x)$ is given by

$$\mu (\theta ,X_{t_{i-1}}):=E\left(X_{t_i}\right|X_{t_{i-1}})=\sum _{k=0}^p\frac{h^k}{k!}{L}^{k}f\left(X_{t_{i-1}}\right)+R(\theta ,h^{p+1},X_{t_{i-1}})$$

where R denotes a function for which there exists a constant C such that

$$R(\theta ,h^{p+1},X_{t_{i-1}})\le h^{p+1}C(1+|X_{t_{i-1}}{\left|\right)}^C.$$

Proof. 

Applying the stochastic Taylor formula of Florens-Zmirou (1989, Lemma 1) [6], one obtains the result. See also Kloeden and Platen (1992) [24].

Consider the following special cases:

Euler Scheme: For $p=1$, $\mu (\theta ,x)=L^{0}f\left(x\right)+h{L}^{1}f\left(x\right)+R(\theta ,h^2,x).$

Milstein Scheme: For $p=2$, $\mu (\theta ,x)=L^{0}f\left(x\right)+h{L}^{1}f\left(x\right)+\frac{h^2}{2!}{L}^{2}f\left(x\right)+R(\theta ,h^3,x).$

Simpson Scheme: For $p=4$, $\mu (\theta ,x)=L^{0}f\left(x\right)+h{L}^{1}f\left(x\right)+\frac{h^2}{2!}{L}^{2}f\left(x\right)+\frac{h^3}{3!}{L}^{3}f\left(x\right)+\frac{h^4}{4!}{L}^{4}f\left(x\right)+R(\theta ,h^5,x).$

Boole Scheme: For $p=6$,

$\mu (\theta ,x)=L^{0}f\left(x\right)+h{L}^{1}f\left(x\right)+\frac{h^2}{2!}{L}^{2}f\left(x\right)+\frac{h^3}{3!}{L}^{3}f\left(x\right)+\frac{h^4}{4!}{L}^{4}f\left(x\right)+\frac{h^5}{5!}{L}^{5}f\left(x\right)+$$\frac{h^6}{6!}{L}^{6}f\left(x\right)+R(\theta ,h^7,x).$ □

Remark 1.

For $p=1$, $\mu (\theta ,X_{t_{i-1}})\approx X_{t_{i-1}}+hf(\theta ,X_{t_{i-1}}).$ This produces the CLSE. This estimator has been very well studied in the literature (see Shoji (1997) [10]).

Remark 2.

Note that the Milstein scheme is equivalent to Stratonovich approximation of the stochastic integral after converting the Itô integral to the Stratonovich integral.

Lemma 3.

For all $p\ge 2$, we have

$$E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^n[\mu (\theta ,X_{t_{i-1}})-X_{t_{i-1}}]\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt\right|}^2\le C\left(\frac{T^{p+1}}{n^{p-1}}\right).$$

Proof. 

First, we show that, for $p=2$,

$$E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt\right|}^2\le C\frac{T^4}{n^2}.$$

We emphasize that the Itô formula is a stochastic Taylor formula of order 2. By the Itô formula, we have

$$\begin{array}{ccc}& & {\displaystyle f({\theta }_0,X_t)-f({\theta }_0,X_{t_{i-1}})}\hfill \\ & =& {\int }_{t_{i-1}}^t\dot{f}({\theta }_0,X_u)d{X}_u+\frac{1}{2}{\int }_{t_{i-1}}^t\ddot{f}({\theta }_0,X_u)du\hfill \\ & =& {\displaystyle {\int }_{t_{i-1}}^t\dot{f}({\theta }_0,X_u)d{W}_u+{\int }_{t_{i-1}}^t[\dot{f}({\theta }_0,X_u)f({\theta }_0,X_u)+\frac{1}{2}\ddot{f}({\theta }_0,X_u)]du}\hfill \\ & =:& {\displaystyle {\int }_{t_{i-1}}^t\dot{f}({\theta }_0,X_u)d{W}_u+{\int }_{t_{i-1}}^{t}F({\theta }_0,X_u)du}\hfill \end{array}$$

where

$$F({\theta }_0,X_u)=:\dot{f}({\theta }_0,X_u)f({\theta }_0,X_u)+\frac{1}{2}\ddot{f}({\theta }_0,X_u).$$

We employ Taylor expansion in the local neighborhood of ${\theta }_0$. Let $\theta ={\theta }_0+T^{-1/2}u,u\in ℝ$. Then, we have

$$\begin{array}{ccc}& & E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt\right|}^2\hfill \\ & =& {\displaystyle E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}\left[f(\theta ,X_t)-f(\theta ,X_{t_{i-1}})\right]dt\right|}^2}\hfill \\ & =& {\displaystyle E\underset{u\in ℝ}{sup}{\left|\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}\left[f({\theta }_0+T^{-1/2}u,X_t)-f({\theta }_0+T^{-1/2}u,X_{t_{i-1}})\right]dt\right|}^2}\hfill \\ & =& {\displaystyle E\underset{u\in ℝ}{sup}\left|\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}\left[f({\theta }_0,X_t)-f({\theta }_0,X_{t_{i-1}})\right]dt\right.}\hfill \\ & & {\left.+T^{-1/2}u\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}\left[f^{\prime }(\overline{\theta },X_t)-f^{\prime }(\overline{\theta },X_{t_{i-1}})\right]dt\right|}^2\hfill \\ & =& 2E{\left|\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}\left[f({\theta }_0,X_t)-f({\theta }_0,X_{t_{i-1}})\right]dt\right|}^2\hfill \\ & & {\displaystyle +2E\underset{u\in ℝ}{sup}{\left|T^{-1/2}u\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}\left[f^{\prime }(\overline{\theta },X_t)-f^{\prime }(\overline{\theta },X_{t_{i-1}})\right]dt\right|}^2}\hfill \\ & =:& 2E{\left|\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}\left[f({\theta }_0,X_t)-f({\theta }_0,X_{t_{i-1}})\right]dt\right|}^2+2G_1\hfill \\ & =& {\displaystyle E{\left|{\displaystyle \sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}\left[{\int }_{t_{i-1}}^t\dot{f}({\theta }_0,X_u)d{W}_u+{\int }_{t_{i-1}}^t{f}^{\prime }({\theta }_0,X_{t_{i-1}})F({\theta }_0,X_u)du\right]dt}\right|}^2+2G_1}\hfill \\ & \le & {\displaystyle 2E{\left|{\displaystyle \sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}{\int }_{t_{i-1}}^t\dot{f}({\theta }_0,X_u)d{W}_{u}dt}\right|}^2}\hfill \\ & & +2E{\left|{\displaystyle \sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}{\int }_{t_{i-1}}^t{f}^{\prime }({\theta }_0,X_{t_{i-1}})F({\theta }_0,X_u)dudt}\right|}^2+2G_1\hfill \\ & =:& {\displaystyle 2(J_1+J_2)+2G_1}\hfill \end{array}$$

where

$$J_1=:E{\left|{\displaystyle \sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}{\int }_{t_{i-1}}^t\dot{f}({\theta }_0,X_u)d{W}_{u}dt}\right|}^2,$$

$$J_2=:E{\left|{\displaystyle \sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}{\int }_{t_{i-1}}^t{f}^{\prime }({\theta }_0,X_{t_{i-1}})F({\theta }_0,X_u)dudt}\right|}^2,$$

$${\displaystyle G_1=:E\underset{u\in ℝ}{sup}{\left|T^{-1/2}u\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}\left[f^{\prime }(\overline{\theta },X_t)-f^{\prime }(\overline{\theta },X_{t_{i-1}})\right]dt\right|}^2}$$

and $|\overline{\theta }-{\theta }_0|\le |\theta -{\theta }_0|$. Further

$$\begin{array}{ccc}& & E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^n[\mu (\theta ,X_{t_{i-1}})-X_{t_{i-1}}]\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt\right|}^2\hfill \\ & \le & 2E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt\right|}^2\hfill \\ & & +2E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^n[\mu (\theta ,X_{t_{i-1}})-X_{t_{i-1}}]\Delta t_i-\sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})\Delta t_i\right|}^2.\hfill \end{array}$$

By Lemma 2, we have

$$\begin{array}{ccc}& & \mu (\theta ,X_{t_{i-1}})-X_{t_{i-1}}-hf(\theta ,X_{t_{i-1}})=\sum _{k=2}^p\frac{h^k}{k!}{L}^{k}f(\theta ,X_{t_{i-1}})+R(\theta ,h^{p+1},X_{t_{i-1}}).\hfill \end{array}$$

Further

$$\begin{array}{ccc}& & \sum _{i=1}^n[\mu (\theta ,X_{t_{i-1}})-X_{t_{i-1}}]\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt\hfill \\ & =& \sum _{i=1}^n[\mu (\theta ,X_{t_{i-1}})-X_{t_{i-1}}]\Delta t_i-\sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})\Delta t_i+\sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt\hfill \\ & =& \sum _{i=1}^n\left[\sum _{k=2}^p\frac{h^k}{k!}{L}^{k}f(\theta ,X_{t_{i-1}})+R(\theta ,h^{p+1},X_{t_{i-1}})\right]+\sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt.\hfill \end{array}$$

Hence

$$\begin{array}{ccc}& & E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^n[\mu (\theta ,X_{t_{i-1}})-X_{t_{i-1}}]\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt\right|}^2\hfill \\ & \le & 4(J_1+J_2)+2E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^n\left[\sum _{k=2}^p\frac{h^k}{k!}{L}^{k}f(\theta ,X_{t_{i-1}})+R(\theta ,h^{p+1},X_{t_{i-1}})\right]\right|}^2.\hfill \end{array}$$

Observe that, with $B_{i,t}:={\int }_{t_{i-1}}^t{f}^{\prime }({\theta }_0,X_{t_{i-1}})\dot{f}({\theta }_0,X_u)d{W}_u,1\le i\le n,$ we have

$$\begin{array}{ccc}{J}_1\hfill & =& {\displaystyle \sum _{i=1}^{n}E{\left({\int }_{t_{i-1}}^{t_i}{B}_{i,t}dt\right)}^2+\sum _{j\ne i=1}^{n}E\left({\int }_{t_{i-1}}^{t_i}{B}_{i,t}dt\right)\left({\int }_{t_{j-1}}^{t_j}{B}_{j,t}dt\right)}\hfill \\ & \le & {\displaystyle \sum _{i=1}^n(t_i-t_{i-1}){\int }_{t_{i-1}}^{t_i}E\left(B_{i,t}^2\right)dt}\hfill \\ & & \left(\mathrm{the\; last\; term\; being\; zero\; due\; to\; the\; orthogonality\; of\; the\; integrals}\right)\hfill \\ & \le & {\displaystyle \sum _{i=1}^n(t_i-t_{i-1}){\int }_{t_{i-1}}^{t_i}\left\{{\int }_{t_{i-1}}^{t}E{\left[f^{\prime }({\theta }_0,X_{t_{i-1}})\dot{f}({\theta }_0,X_u)\right]}^{2}du\right\}dt}\hfill \\ & \le & {\displaystyle C\frac{T}{n}\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}(t-t_{i-1})dt(\mathrm{by} (\mathrm{A}4) \mathrm{and} (\mathrm{A}3\left)\right)}\hfill \\ & \le & {\displaystyle C\frac{T}{n}\sum _{i=1}^n{(t_i-t_{i-1})}^2}\hfill \\ & =& {\displaystyle C\frac{T^3}{n^2}.}\hfill \end{array}$$

On the other hand, with $A_{i,t}:={\int }_{t_{i-1}}^t{f}^{\prime }({\theta }_0,X_{t_{i-1}})F({\theta }_0,X_u)du,1\le i\le n,$ we have

$$\begin{array}{ccc}{J}_2\hfill & =& {\displaystyle E{\left|\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}{\int }_{t_{i-1}}^t{f}^{\prime }({\theta }_0,X_{t_{i-1}})F({\theta }_0,X_u)dudt\right|}^2}\hfill \\ & =& {\displaystyle E{\left|\sum _{i=1}^n{\int }_{t_{i-1}}^{t_i}{A}_{i,t}dt\right|}^2}\hfill \\ & =& {\displaystyle \sum _{i=1}^{n}E{\left({\int }_{t_{i-1}}^{t_i}{A}_{i,t}dt\right)}^2+\sum _{j\ne i=1}^{n}E\left({\int }_{t_{i-1}}^{t_i}{A}_{i,t}dt\right)\left({\int }_{t_{j-1}}^{t_j}{A}_{j,t}dt\right)}\hfill \\ & \le & {\displaystyle \sum _{i=1}^n(t_i-t_{i-1})E\left({\int }_{t_{i-1}}^{t_i}{A}_{i,t}^{2}dt\right)+\sum _{j\ne i=1}^n{\left\{E{\left({\int }_{t_{i-1}}^{t_i}{A}_{i,t}dt\right)}^{2}E{\left({\int }_{t_{j-1}}^{t_j}{A}_{j,t}dt\right)}^2\right\}}^{1/2}}\hfill \\ & \le & {\displaystyle \sum _{i=1}^n(t_i-t_{i-1}){\int }_{t_{i-1}}^{t_i}E\left(A_{i,t}^2\right)dt}\hfill \\ & & {\displaystyle +\sum _{j\ne i=1}^n{\left\{(t_i-t_{i-1}){\int }_{t_{i-1}}^{t_i}E\left(A_{i,t}^2\right)dt(t_j-t_{j-1}){\int }_{t_{j-1}}^{t_j}E\left(A_{j,t}^2\right)dt\right\}}^{1/2}.}\hfill \end{array}$$

However, $E\left(A_{i,t}^2\right)\le C{(t-t_{i-1})}^2$ using (A4) and (A3). On substitution, the last term is dominated by

$$\begin{array}{ccc}& & {\displaystyle C\sum _{i=1}^n{(t_i-t_{i-1})}^4+C\sum _{j\ne i=1}^n{(t_i-t_{i-1})}^2{(t_j-t_{j-1})}^2}\hfill \\ & =& {\displaystyle C\frac{T^4}{n^3}+C\frac{n(n-1)T^4}{2n^4}\le C\frac{T^4}{n^2}.}\hfill \end{array}$$

Thus

$$J_1+J_2\le C\frac{T^4}{n^2}.$$

By the same method, we have

$$G_1\le C\frac{T^3}{n^2}.$$

Hence

$$\begin{array}{ccc}& & {\displaystyle E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt\right|}^2}\hfill \\ & & {\displaystyle \le 2(J_1+J_2)+2G_1}\hfill \\ & & {\displaystyle \le C\frac{T^4}{n^2}.}\hfill \end{array}$$

Thus, the proof for $p=2$ is complete. Next, we consider the general case $p\ge 3$. Denote

$$J_3:=E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^n\left[\sum _{k=2}^p\frac{h^k}{k!}{L}^{k}f(\theta ,X_{t_{i-1}})+R(\theta ,h^{p+1},X_{t_{i-1}})\right]\right|}^2.$$

We have

$$\begin{array}{ccc}& & {\displaystyle E\underset{\theta \in \Theta }{sup}{\left|{\displaystyle \sum _{i=1}^n[\mu (\theta ,X_{t_{i-1}})-X_{t_{i-1}}]\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt}\right|}^2}\hfill \\ & \le & {\displaystyle 2E\underset{\theta \in \Theta }{sup}{\left|{\displaystyle \sum _{i=1}^n[\mu (\theta ,X_{t_{i-1}})-X_{t_{i-1}}]\Delta t_i-\sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})\Delta t_i}\right|}^2}\hfill \\ & & {\displaystyle +2E\underset{\theta \in \Theta }{sup}{\left|{\displaystyle \sum _{i=1}^{n}f(\theta ,X_{t_{i-1}})\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt}\right|}^2}\hfill \\ & \le & 2J_3+4(J_1+J_2)+2G_1.\hfill \end{array}$$

Observe that, by Lemma 2, we have

$$J_3\le C\frac{T^{p+1}}{n^{p-1}}.$$

Thus, by combining the bounds for $J_1,J_2$, $J_3$ and $G_1$, we have

$$E\underset{\theta \in \Theta }{sup}{\left|\sum _{i=1}^n[\mu (\theta ,X_{t_{i-1}})-X_{t_{i-1}}]\Delta t_i-{\int }_0^{T}f(\theta ,X_t)dt\right|}^2\le C\left(\frac{T^{p+1}}{n^{p-1}}\right).$$

□

The following lemma is from Bishwal (2008) [1].

Lemma 4.

Let

$$I_T:=\frac{1}{TI\left({\theta }_0\right)}{\int }_0^T{f}^2({\theta }_0,X_t)dt.$$

Then, under the conditions (A1)–(A8),

$$\underset{\theta \in \Theta }{sup}E{[I_T\left(\theta \right)-1]}^2\le C{T}^{-1}.$$

The following lemma follows from Theorem 7 in Yoshida (1997) [22].

Lemma 5.

Let

$$M_T:=\frac{1}{\sqrt{TI\left({\theta }_0\right)}}{\int }_0^{T}f({\theta }_0,X_t)d{W}_t.$$

Then, under the conditions (A1)–(A8),

$${\displaystyle \underset{x\in ℝ}{sup}\left|P_{{\theta }_0}\left\{M_T\le x\right\}-\Phi \left(x\right)\right|\le C{T}^{-1/2}.}$$

Our main result is the following theorem.

Theorem 1.

Under the conditions (A1)-(A8), for any $p\ge 1$, we have

$${\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{\sqrt{TI\left(\theta \right)}({\theta }_{n,T}-\theta )\le x\right\}-\Phi \left(x\right)\right|=O\left(T^{-1/2}\bigvee \frac{T^{p+1}}{n^p}\right).}$$

Proof. 

We start with $p=1$ and $p=2.$ Let

$${\widehat{l}}_{n,T}\left(\theta \right):=log{\widehat{L}}_{n,T}\left(\theta \right),and{\tilde{l}}_{n,T}\left(\theta \right):=log{\tilde{L}}_{n,T}\left(\theta \right).$$

By Taylor expansion, we have

$${\widehat{l}}_{n,T}^{\prime }\left({\widehat{\theta }}_{n,T}\right)={\widehat{l}}_{n,T}^{\prime }\left({\theta }_0\right)+({\widehat{\theta }}_{n,T}-{\theta }_0){\widehat{l}}_{n,T}^{″}\left({\overline{\theta }}_{n,T}\right)$$

where $\left|{\overline{\theta }}_{n,T}-\theta \right|\le \left|{\widehat{\theta }}_{n,T}-{\theta }_0\right|$. Since ${\widehat{l}}_{n,T}^{\prime }\left({\widehat{\theta }}_{n,T}\right)=0$, hence we have

$$\begin{array}{c}\hfill {\displaystyle \sqrt{TI\left({\theta }_0\right)}({\widehat{\theta }}_{n,T}-{\theta }_0)=-\frac{\frac{1}{\sqrt{TI\left({\theta }_0\right)}}{\widehat{l}}_{n,T}^{\prime }\left({\theta }_0\right)}{\frac{1}{TI\left({\theta }_0\right)}{\widehat{l}}_{n,T}^{″}\left({\overline{\theta }}_{n,T}\right)}=-\frac{\frac{1}{\sqrt{TI\left({\theta }_0\right)}}{\sum }_{i=1}^n{f}^{\prime }({\theta }_0,X_{t_{i-1}})\Delta W_i}{\frac{1}{TI\left({\theta }_0\right)}{\sum }_{i=1}^n{f}^{″}({\overline{\theta }}_{n,T},X_{t_{i-1}})\Delta t_i}=:\frac{M_{n,T}}{V_{n,T}}}\end{array}$$

Note that

$$V_{n,T}=\frac{1}{TI\left({\theta }_0\right)}\sum _{i=1}^n{f}^{″}({\overline{\theta }}_{n,T},X_{t_{i-1}})\Delta t_i=\frac{1}{TI\left({\theta }_0\right)}\sum _{i=1}^n{f}^{\prime }{({\overline{\theta }}_{n,T},X_{t_{i-1}})}^2\Delta t_i.$$

However, $E{(I_T-1)}^2\le C{T}^{-1}$ from Lemma 4 (see also Pardoux and Veretennikov (2001) [25] and Yoshida (2011) [26]). It can be shown that $E{(V_{n,T}-I_T)}^2\le C\frac{T}{n}$ (see Altmeyer and Chorowski (2018) [27]). Hence

$$E{(V_{n,T}-1)}^2=E{[(V_{n,T}-I_T)+(I_T-1)]}^2\le C(T^{-1}\bigvee \frac{T}{n}).$$

Further, by Lemma 1 (b), we have

$$\begin{array}{ccc}& & {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{\sqrt{TI\left(\theta \right)}({\widehat{\theta }}_{n,T}-\theta )\le x\right\}-\Phi \left(x\right)\right|}\hfill \\ & =& {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{\frac{M_{n,T}}{V_{n,T}}\le x\right\}-\Phi \left(x\right)\right|}\hfill \\ & =& {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{M_{n,T}\le x\right\}-\Phi \left(x\right)\right|+P_{\theta }\left\{\left|V_{n,T}-1\right|\ge ϵ\right\}+ϵ}\hfill \\ & \le & C(T^{-1/2}\bigvee \frac{T^2}{n})+{ϵ}^{-2}C(T^{-1}\bigvee \frac{T}{n})+ϵ.\hfill \end{array}$$

since, by Lemmas 1 (a) and 5, we have

$$\begin{array}{ccc}& & {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{M_{n,T}\le x\right\}-\Phi \left(x\right)\right|}\hfill \\ & \le & {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{M_T\le x\right\}-\Phi \left(x\right)\right|+P_{\theta }\left\{\left|M_{n,T}-M_T\right|\ge ϵ\right\}+ϵ}\hfill \\ & \le & C{T}^{-1/2}+{ϵ}^{-2}E{\left|M_{n,T}-M_T\right|}^2+ϵ\hfill \\ & \le & C(T^{-1/2}\bigvee \frac{T^2}{n})+{ϵ}^{-2}C\frac{T}{n}+ϵ.\hfill \end{array}$$

Choosing $ϵ=T^{-1/2}$, we have the result.

On the other hand, by Taylor expansion, we have

$${\tilde{l}}_{n,T}^{\prime }\left({\tilde{\theta }}_{n,T}\right)={\tilde{l}}_{n,T}^{\prime }\left({\theta }_0\right)+({\tilde{\theta }}_{n,T}-{\theta }_0){\tilde{l}}_{n,T}^{″}\left({\overline{\overline{\theta }}}_{n,T}\right)$$

where $\left|{\overline{\overline{\theta }}}_{n,T}-\theta \right|\le \left|{\tilde{\theta }}_{n,T}-{\theta }_0\right|.$ Since ${\tilde{l}}_{n,T}^{\prime }\left({\tilde{\theta }}_{n,T}\right)=0$, hence we have

$$\begin{array}{ccc}& & \sqrt{TI\left({\theta }_0\right)}({\tilde{\theta }}_{n,T}-{\theta }_0)\hfill \\ & =& -\frac{\frac{1}{\sqrt{TI\left({\theta }_0\right)}}{\tilde{l}}_{n,T}^{\prime }\left({\theta }_0\right)}{\frac{1}{TI\left({\theta }_0\right)}{\tilde{l}}_{n,T}^{″}\left({\overline{\overline{\theta }}}_{n,T}\right)}\hfill \\ & =& -\left\{\frac{1}{\sqrt{TI\left({\theta }_0\right)}}\left\{\frac{1}{2}\sum _{i=1}^n[f^{\prime }({\theta }_0,X_{t_{i-1}})+f^{\prime }({\theta }_0,X_{t_i})]\Delta W_i\right.\right.\hfill \\ & & +\frac{1}{2}\sum _{i=1}^n[f^{\prime }({\theta }_0,X_{t_{i-1}})+f^{\prime }({\theta }_0,X_{t_i})]{\int }_{t_{i-1}}^{t_i}{f}^{\prime }({\theta }_0,X_t)dt\hfill \\ & & \left.\left.-\frac{h}{2}\sum _{i=1}^n{\dot{f}}^{\prime }({\theta }_0,X_{t_{i-1}})-h\sum _{i=1}^{n}f({\theta }_0,X_{t_{i-1}})f^{\prime }({\theta }_0,X_{t_{i-1}})\right\}\right\}\hfill \\ & & \times \left\{\frac{1}{TI\left({\theta }_0\right)}\left\{\frac{1}{2}\sum _{i=1}^n[f^{″}({\overline{\overline{\theta }}}_{n,T},X_{t_{i-1}})+f^{″}({\overline{\overline{\theta }}}_{n,T},X_{t_{i-1}})]\Delta W_i\right.\right.\hfill \\ & & -\frac{1}{2}\sum _{i=1}^n[f^{\prime }({\overline{\overline{\theta }}}_{n,T},X_{t_{i-1}})+f^{\prime }({\overline{\overline{\theta }}}_{n,T},X_{t_i})]{\int }_{t_{i-1}}^{t_i}{f}^{\prime }({\overline{\overline{\theta }}}_{n,T},X_t)dt\hfill \end{array}$$

$$\begin{array}{ccc}& & -\frac{h}{2}\sum _{i=1}^n{\dot{f}}^{″}({\overline{\overline{\theta }}}_{n,T},X_{t_{i-1}})-h\sum _{i=1}^{n}f({\overline{\overline{\theta }}}_{n,T},X_{t_{i-1}})f^{″}({\overline{\overline{\theta }}}_{n,T},X_{t_{i-1}})\hfill \\ & & {\left.\left.-h\sum _{i=1}^n{f}^{\prime 2}({\overline{\overline{\theta }}}_{n,T},X_{t_{i-1}})\right\}\right\}}^{-1}\hfill \\ & =:& \left\{R_{n,T}\right\}{\left\{S_{n,T}\right\}}^{-1}.\hfill \end{array}$$

Let $lim{S}_{n,T}=S_T$ in $L_2$ as $T\to \infty$ and $\frac{T}{n}\to 0$. Similar to Lemma 4, it can be shown that $E{(S_T-1)}^2\le C{T}^{-1}$ (see also Pardoux and Veretennikov (2001) [25] and Yoshida (2011) [26]). It can be shown that $E{(S_{n,T}-S_T)}^2\le C\frac{T}{n}$ (see Altmeyer and Chorowski (2018) [27]). Hence

$$E{(S_{n,T}-1)}^2=E{[(S_{n,T}-S_T)+(S_T-1)]}^2\le C(T^{-1}\bigvee \frac{T}{n}).$$

Thus, by Lemma 1 (b), we have

$$\begin{array}{ccc}& & {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{\sqrt{TI\left(\theta \right)}({\tilde{\theta }}_{n,T}-\theta )\le x\right\}-\Phi \left(x\right)\right|}\hfill \\ & =& {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{\frac{R_{n,T}}{S_{n,T}}\le x\right\}-\Phi \left(x\right)\right|}\hfill \\ & =& {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{R_{n,T}\le x\right\}-\Phi \left(x\right)\right|+P_{\theta }\left\{\left|S_{n,T}-1\right|\ge ϵ\right\}+ϵ}\hfill \\ & \le & C(T^{-1/2}\bigvee \frac{T^3}{n^2})+{ϵ}^{-2}C(T^{-1}\bigvee \frac{T}{n})+ϵ.\hfill \end{array}$$

since, by Lemmas 1 (a) and 5, we have

$$\begin{array}{ccc}& & {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{R_{n,T}\le x\right\}-\Phi \left(x\right)\right|}\hfill \\ & \le & {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{M_T\le x\right\}-\Phi \left(x\right)\right|+P_{\theta }\left\{\left|R_{n,T}-M_T\right|\ge ϵ\}+ϵ\right.}\hfill \\ & \le & C{T}^{-1/2}+{ϵ}^{-2}E{\left|R_{n,T}-M_T\right|}^2+ϵ\hfill \\ & \le & C{T}^{-1/2}+{ϵ}^{-2}C\frac{T^3}{n^2}+ϵ.\hfill \end{array}$$

Choosing $ϵ=T^{-1/2}$, we have the result.

Now, we study the general case for arbitrary p. By Taylor expansion, we have

$$K_{n,T}^{\prime }\left({\theta }_{n,T}\right)=K_{n,T}^{\prime }\left({\theta }_0\right)+({\theta }_{n,T}-{\theta }_0)K_{n,T}^{″}\left({\overline{\overline{\overline{\theta }}}}_{n,T}\right)$$

where $\left|{\overline{\overline{\overline{\theta }}}}_{n,T}-\theta \right|\le \left|{\theta }_{n,T}-{\theta }_0\right|$. Since $K_{n,T}^{\prime }\left({\theta }_{n,T}\right)=0$, hence we have

$$\begin{array}{c}\hfill {\displaystyle \sqrt{TI\left({\theta }_0\right)}({\theta }_{n,T}-{\theta }_0)=-\frac{\frac{1}{\sqrt{TI\left({\theta }_0\right)}}{K}_{n,T}^{\prime }\left({\theta }_0\right)}{\frac{1}{TI\left({\theta }_0\right)}{K}_{n,T}^{″}\left({\overline{\overline{\overline{\theta }}}}_{n,T}\right)}=-\frac{\frac{1}{\sqrt{TI\left({\theta }_0\right)}}{\sum }_{i=1}^n{m}^{\prime }({\theta }_0,X_{t_{i-1}})\Delta W_i}{\frac{1}{TI\left({\theta }_0\right)}{\sum }_{i=1}^n{m}^{″}({\overline{\overline{\overline{\theta }}}}_{n,T},X_{t_{i-1}})\Delta t_i}=:\frac{N_{n,T}}{U_{n,T}}}\end{array}$$

Note that

$$U_{n,T}=\frac{1}{TI\left({\theta }_0\right)}\sum _{i=1}^n{m}^{″}({\overline{\overline{\overline{\theta }}}}_{n,T},X_{t_{i-1}})\Delta t_i=\frac{1}{TI\left({\theta }_0\right)}\sum _{i=1}^n{m}^{\prime }{({\overline{\overline{\overline{\theta }}}}_{n,T},X_{t_{i-1}})}^2\Delta t_i.$$

Let $lim{U}_{n,T}=U_T$ in $L_2$ as $T\to \infty$ and $\frac{T}{n}\to 0$. Similar to Lemma 4, it can be shown that $E{(U_T-1)}^2\le C{T}^{-1}$ (see also Pardoux and Veretennikov (2001) [25] and Yoshida (2011) [26]). It can be shown that $E[{(U_{n,T}-U_T)}^2\le C\frac{T}{n}$ (see Altmeyer and Chorowski (2018) [27]). Hence

$$E{(U_{n,T}-1)}^2=E{[(U_{n,T}-U_T)+(U_T-1)]}^2\le C(T^{-1}\bigvee \frac{T}{n}).$$

Further, by Lemma 1 (b), we have

$$\begin{array}{ccc}& & {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{\sqrt{TI\left(\theta \right)}({\theta }_{n,T}-\theta )\le x\right\}-\Phi \left(x\right)\right|}\hfill \\ & =& {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{\frac{N_{n,T}}{U_{n,T}}\le x\right\}-\Phi \left(x\right)\right|}\hfill \\ & =& {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{N_{n,T}\le x\right\}-\Phi \left(x\right)\right|+P_{\theta }\left\{\left|U_{n,T}-1\right|\ge ϵ\right\}+ϵ}\hfill \\ & \le & C(T^{-1/2}\bigvee \frac{T^{p+1}}{n^p})+{ϵ}^{-2}C(T^{-1}\bigvee \frac{T}{n})+ϵ.\hfill \end{array}$$

since, by Lemmas 1 (a) and 5, we have

$$\begin{array}{ccc}& & {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{N_{n,T}\le x\right\}-\Phi \left(x\right)\right|}\hfill \\ & \le & {\displaystyle \underset{x\in ℝ}{sup}\left|P_{\theta }\left\{M_T\le x\right\}-\Phi \left(x\right)\right|+P_{\theta }\left\{\left|N_{n,T}-M_T\right|\ge ϵ\right\}+ϵ}\hfill \\ & \le & C{T}^{-1/2}+{ϵ}^{-2}E{\left|N_{n,T}-M_T\right|}^2+ϵ\hfill \\ & \le & C{T}^{-1/2}+{ϵ}^{-2}C\frac{T^{p+1}}{n^p}+ϵ.\hfill \end{array}$$

Choosing $ϵ=T^{-1/2}$, we have the result. □

Remark 3.

With $p=1$, for the Euler scheme, which produces the conditional least squares estimator, one obtains the rate $O\left(T^{-1/2}\bigvee \frac{T^2}{n}\right).$ With $p=2$, for the Milstein scheme, one obtains the rate $O\left(T^{-1/2}\bigvee \frac{T^3}{n^2}\right).$ With $p=4$, for the Simpson scheme, one obtains the rate $O\left(T^{-1/2}\bigvee \frac{T^5}{n^4}\right).$ With $p=6$, for the Boole scheme, one obtains the rate $O\left(T^{-1/2}\bigvee \frac{T^7}{n^6}\right).$ Thus, the higher the p, the sharper the bound. Thus, the Itô/Euler scheme gives the first-order QMLE, the Milstein/Stratonovich scheme produces the second-order QMLE, the Simpson scheme produces the fourth-order QMLE and the Boole scheme produces the sixth-order QMLE. See Bishwal (2011) [28] for a connection of this area to the stochastic moment problem and hedging of generalized Black–Scholes options.

3. Example

Consider the stochastic differential equation

$$\begin{array}{ccc}d{X}_t\hfill & =& {\displaystyle \theta \frac{X_t}{\sqrt{1+X_t^2}}dt+d{W}_t,t\ge 0}\hfill \\ X_0\hfill & =& {\displaystyle x_0}\hfill \end{array}$$

The solution to the above SDE is called the hyperbolic diffusion process because it has a hyperbolic stationary distribution when $\theta <0$. The process has nonlinear drift and the process is stationary and ergodic, which distinguishes this from a linear drift case, such as the Ornstein–Uhlenbeck process and the Cox–Ingersoll–Ross process, which have linear drift. This model verifies assumption (A3). In fact, the stationary density is proportional to $exp\left(\theta \sqrt{1+X_t^2}\right)$. It is not possible to calculate the conditional expectation for the hyperbolic diffusion process and hence one needs a higher-order Taylor expansion approach.

Remark 4

(Concluding Remark). It would be interesting to extend the results of the paper to diffusions with jumps using the strong stochastic Taylor expansion with jumps results in Chapter 6 of Kloeden and Bruti-Liberati (2010) [29].