Convergence Rate of Runge-Kutta-Type Regularization for Nonlinear Ill-Posed Problems under Logarithmic Source Condition

Abstract

We prove the logarithmic convergence rate of the families of usual and modified iterative Runge-Kutta methods for nonlinear ill-posed problems between Hilbert spaces under the logarithmic source condition, and numerically verify the obtained results. The iterative regularization is terminated by the a posteriori discrepancy principle.

1. Introduction

Let X and Y be infinite-dimensional real Hilbert spaces with inner products $⟨·,·⟩$ and norms $\parallel ·\parallel$. Let us consider a nonlinear ill-posed operator equation

$$F\left(w\right)=g,$$

(1)

where $F:D\left(F\right)\subset X\to Y$ is a nonlinear operator between the Hilbert spaces X and Y. We assume that (1) has a solution $w^{+}$ for exact data (which need not be unique). We have approximate data $g^{\epsilon }$ with

$$\parallel g^{\epsilon }-g\parallel \le \epsilon ,\epsilon >0.$$

(2)

Besides the classical Tikhonov–Phillips regularization, a plethora of interesting variational and iterative approaches for ill-posed problems can be found, e.g., in Morozov [1], Tikhonov and Arsenin [2], Bakushinsky and Kokurin [3], and Kaltenbacher et al. [4]. We focus here on iterative methods, as they are also very popular and effective to use in applications. The simplest iterative regularization is the Landweber method—see, e.g., Hanke et al. [5], where the analysis for convergence rates is done under Hölder-type source condition. A more effective method often used in applications is the Levenberg–Marquardt method

$$w_{k+1}^{\epsilon }=w_k^{\epsilon }+{({\alpha }_{k}I+F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}{F}^{\prime }\left(w_k^{\epsilon }\right))}^{-1}{F}^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(g^{\epsilon }-F\left(w_k^{\epsilon }\right)).$$

(3)

This was investigated in [6,7,8] under the Hölder-type source condition (HSC) and a posteriori discrepancy principle (DP). Jin [7] proved optimal convergence rates for an a priori chosen geometric step size sequence ${\alpha }_k$, whereas Hochbruck and Hönig [6] showed convergence with the optimal rate for quite general step size sequences including the geometric sequence. Later, Hanke [8] avoided any constraints on the rate of decay of the regularization parameter to show the optimal convergence rate.

Tautenhahn [9] proved that asymptotic regularization, i.e., the approximation of problem (1) by a solution of the Showalter differential equation (SDE)

$$\begin{array}{c}\hfill \frac{d}{dt}{w}^{\epsilon }\left(t\right)=F^{\prime }{\left(w^{\epsilon }\left(t\right)\right)}^{*}[g^{\epsilon }-F\left(w^{\epsilon }\left(t\right)\right)],0<t\le T,w^{\epsilon }\left(0\right)=\overline{w},\end{array}$$

(4)

where the regularization parameter T is chosen according to the DP under the HSC, is a stable method to solve nonlinear ill-posed problems.

Solving SDE by the family of Runge-Kutta (RK) methods delivers a family of RK-type iterative regularization methods

$$w_{k+1}^{\epsilon }=w_k^{\epsilon }+{\tau }_k{b}^T{(\delta +{\tau }_{k}A{F}^{\prime }{\left(w_k^{\epsilon }\right)}^{*}{F}^{\prime }\left(w_k^{\epsilon }\right))}^{-1}𝟙F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(g^{\epsilon }-F\left(w_k^{\epsilon }\right)),k\in {\mathbb{N}}_0,$$

(5)

where $𝟙$ denotes the $(s\times 1)$ vector of identity operators, while $\delta$ is the $(s\times s)$ diagonal matrix of bounded linear operators with identity operator on the entire diagonal and zero operator outside of the main diagonal with respect to the appropriate spaces. The parameter ${\tau }_k=1/{\alpha }_k$ in (5) is the step-length, also called the relaxation parameter. The $(s\times s)$ matrix A and the $(s\times 1)$ vector b are the given parameters that correspond to the specific RK method, building the so-called Butcher tableau (succession of stages). Different choices of the RK parameters generate various iterative methods.

Böckmann and Pornsawad [10] showed convergence for the whole RK-type family (including the well-known Landweber and the Levenberg–Marquardt methods). That paper also emphasized advantages of using some procedures from the mentioned family, e.g., regarding implicit A-stable Butcher tableaux. For instance, the Landweber method needs a lot of iteration steps, while those implicit methods need only a few iteration steps, thus minimizing the rounding errors.

Later, Pornsawad and Böckmann [11] filled in the missing results on optimal convergence rates, but only for particular first-stage methods under HSC and using DP.

Our current study considers further the unifying RK-framework described above, as well as a modified version presented below, showing optimality of the RK-regularization schemes under logarithmic source conditions.

An additional term ${\alpha }_k(w_k^{\epsilon }-\xi )$, as in the iteratively regularized Gauss–Newton method (see, e.g., [4]),

$$\begin{array}{c}\hfill w_{k+1}^{\epsilon }=w_k^{\epsilon }-{(F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}{F}^{\prime }\left(w_k^{\delta }\right)+{\alpha }_{k}I)}^{-1}(F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(F\left(w_k^{\epsilon }\right)-g^{\epsilon })+{\alpha }_k(w_k^{\epsilon }-\xi )),\end{array}$$

was added to a modified Landweber method. Thus, Scherzer [12] proved a convergence rate result under HSC without particular assumptions on the nonlinearity of operator F. Moreover, in Pornsawad and Böckmann [13], an the additional term was included to the whole family of iterative RK-type methods (which contains the modified Landweber iteration),

$$w_{k+1}^{\epsilon }=w_k^{\epsilon }+{\tau }_k{b}^T{\Pi }^{-1}𝟙F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(g^{\epsilon }-F\left(w_k^{\epsilon }\right))-{\tau }_k^{-1}(w_k^{\epsilon }-\zeta ),$$

(6)

where $\zeta \in X$ and $\Pi =\delta +{\tau }_{k}A{F}^{\prime }{\left(w_k^{\epsilon }\right)}^{*}{F}^{\prime }\left(w_k^{\epsilon }\right)$. Using a priori and a posteriori stopping rules, the convergence rate results of the RK-type family are obtained under HSC if the Fréchet derivative is properly scaled.

Due to the minimal assumptions for the convergence analysis of the modified iterative RK-type methods, an additional term was added to SDE

$$\begin{array}{c}\hfill \frac{d}{dt}{w}^{\epsilon }\left(t\right)=F^{\prime }{\left(w^{\epsilon }\left(t\right)\right)}^{*}[g^{\epsilon }-F\left(w^{\epsilon }\left(t\right)\right)]-(w^{\epsilon }\left(t\right)-\overline{w}),0<t\le T,w^{\epsilon }\left(0\right)=\overline{w}.\end{array}$$

(7)

Pornsawad et al. [14] investigated this continuous version of the modified iterative RK-type methods for nonlinear inverse ill-posed problems. The convergence analysis yields the optimal rate of convergence under a modified DP and an exponential source condition.

Recently, a second-order asymptotic regularization for the linear problem $\widehat{A}x=y$ was investigated in [15]

$$\ddot{x}\left(t\right)+\mu \dot{x}\left(t\right)+{\widehat{A}}^{*}\widehat{A}x\left(t\right)={\widehat{A}}^{*}{y}^{\delta },x\left(0\right)=\overline{x},\dot{x}\left(0\right)=\dot{\overline{x}}$$

under HSC using DP.

Define

$$\phi ={\phi }_p,{\phi }_p\left(\lambda \right):=\left\{\begin{array}{cc}{\left(ln\frac{e}{\lambda }\right)}^{-p}\hfill & \mathrm{for} 0<\lambda \le 1\hfill \\ 0\hfill & \mathrm{for} \lambda =0\hfill \end{array}\right.$$

(8)

with $p>0$ and the usual logarithmic sourcewise representation

$$\begin{array}{c}\hfill w^{+}-w_0=\phi \left(F^{\prime }{\left(w^{+}\right)}^{*}{F}^{\prime }\left(w^{+}\right)\right)v,\hspace{1em}\hspace{1em}\hspace{1em}v\in X,\end{array}$$

(9)

where $∥v∥$ is sufficiently small and $w_0\in D\left(F\right)$ is an initial guess that may incorporate a priori knowledge on the solution.

In numerous applications—e.g., heat conduction, scattering theory, which are severely ill-posed problems—the Hölder source condition is far too strong. Therefore, Hohage [16] proved convergence and logarithmic convergence rates for the iteratively regularized Gauss–Newton method in a Hilbert space setting, provided a logarithmic source condition (9) is satisfied and DP is used as the stopping rule. Deuflhard et al. [17] showed some convergence rate result for the Landweber iteration using DP and (9) under a Newton–Mysovskii condition on the nonlinear operator. Sufficient conditions for the convergence rate, which is logarithmic in the data noise level, are given.

In Hohage [18], a systematic study of convergence rates for regularization methods under (9) including the case of operator approximations for a priori and a posteriori stopping rules is provided. A logarithmic source condition is considered by Pereverzyev et al. [19] for a derivative-free method, by Mahale and Nair [20] for a simplified generalized Gauss–Newton method, and by Böckmann et al. [21] for the Levenberg–Marquardt method using DP as stopping rule. Pornsawad et al. [22] solved the inverse potential problem, which is exponentially ill-posed, employing the modified Landweber method and proved convergence rate under the logarithmic source condition via DP for this method.

To the best of our knowledge, for the first time, convergence rates are established both for the whole family of RK-methods and for the modified version, when applied to severely ill-posed problems (i.e., under the logarithmic source condition).

The structure of this article is as follows. Section 2 provides assumptions and technical estimations. We derive the convergence rate of the RK-type method (5) in Section 3 and of the modified RK-type method (6) in Section 4 under the logarithmic source conditions (8) and (9). In Section 5, the performed numerical experiments confirm the theoretical results.

2. Preliminary Results

Lemma 1.

Let K be a linear operator with $\parallel K\parallel \le 1$. For $k\in N$ with $k>1,e_0:=\phi \left(\lambda \right)v$ with φ given by (8) and $p>0$, there exist positive constants $c_1$ and $c_2$ such that

$$\begin{array}{c}\hfill ∥{(I-K^{*}K)}^k{e}_0∥\le c_1{(ln(k+e))}^{-p}\parallel v\parallel \end{array}$$

(10)

and

$$\begin{array}{c}\hfill ∥K{(I-K^{*}K)}^k{e}_0∥\le c_2{(k+1)}^{-1/2}{(ln(k+e))}^{-p}\parallel v\parallel .\end{array}$$

(11)

Proof. 

By spectral theory (8), and (A1) and (A2) in [22], we have

$$\begin{array}{ccc}\hfill ∥{(I-K^{*}K)}^k{e}_0∥& \le & \parallel {(I-K^{*}K)}^k\phi \left(K^{*}K\right)\parallel \parallel v\parallel \hfill \\ & \le & \underset{\lambda \in (0,1]}{sup}{|(1-\lambda )}^k{(1-ln\lambda )}^{-p}|\parallel v\parallel \hfill \\ & \le & c_1{(ln(k+e))}^{-p}\parallel v\parallel ,\hfill \end{array}$$

(12)

for some constant $c_1>0$. Similarly, spectral theory (8), and (A3) and (A4) in [22], provides

$$\begin{array}{ccc}\hfill ∥K{(I-K^{*}K)}^k{e}_0∥& \le & \parallel {(I-K^{*}K)}^k{\left(K^{*}K\right)}^{1/2}\phi \left(K^{*}K\right)\parallel \parallel v\parallel \hfill \\ & \le & \underset{\lambda \in (0,1]}{sup}{|(1-\lambda )}^k{\lambda }^{1/2}{(1-ln\lambda )}^{-p}|\parallel v\parallel \hfill \\ & \le & c_2{(k+1)}^{-1/2}{(ln(k+e))}^{-p}\parallel v\parallel ,\hfill \end{array}$$

(13)

for some constant $c_2>0$. □

Assumption 1.

There exist positive constants $c_L,c_r$, and ${\widehat{c}}_R$ and a linear bounded operator $R_w:Y\to Y$ such that for $w\in$$B_{\rho }\left(w_0\right)$, the following conditions hold

$$\begin{array}{cc}\hfill F^{\prime }\left(w\right)& =R_w{F}^{\prime }\left(w^{+}\right)\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill ∥R_w-I∥& \le c_L∥w-w^{+}∥\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill \parallel R_w\parallel & \le c_r\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill |\parallel R_w\parallel -\parallel I\parallel |& \ge {\widehat{c}}_R,\hfill \end{array}$$

(17)

where $w^{+}$ is the exact solution of (1).

Let $e_k:=w^{+}-w_k^{\epsilon }$ be the error of the kth iteration $w_k^{\epsilon }$, $S_k:=F^{\prime }\left(w_k^{\epsilon }\right)$ and $S:=F^{\prime }\left(w^{+}\right)$.

Proposition 1.

Let the conditions (14) and (15) in Assumption 1 hold. Then,

$$\begin{array}{c}\hfill ∥F\left(w_k^{\epsilon }\right)-F\left(w^{+}\right)-F^{\prime }\left(w^{+}\right)(w_k^{\epsilon }-w^{+})∥\le \frac{1}{2}{c}_L∥e_k∥∥S{e}_k∥\end{array}$$

(18)

for $w\in$$B_{\rho }\left(w_0\right)$.

The proof is given in [22] using the mean value theorem.

Assumption 2.

Let K be a linear operator and τ be a positive number. There exist positive constants $D_1$ and $D_2$ such that

$$\parallel I-\tau f(-\tau K{K}^{*})\parallel \le D_1$$

(19)

and

$$\parallel I+\tau f(-\tau K{K}^{*})\parallel \le D_2,$$

(20)

with $f\left(t\right):=b^T{(I-At)}^{-1}𝟙$.

We note that the explicit Euler method provides $\parallel I-\tau f(-\tau K{K}^{*})\parallel \le |1-\tau |$ and $\parallel I+\tau f(-\tau K{K}^{*})\parallel \le |1+\tau |$. Thus, the conditions (19) and (20) hold if $\tau$ is bounded. For the implicit Euler method, we have

$$\parallel I-\tau f(-\tau K{K}^{*})\parallel \le \underset{0<\lambda \le {\lambda }_0}{sup}|1-\tau {(1+\tau \lambda )}^{-1}|$$

and

$$\parallel I+\tau f(-\tau K{K}^{*})\parallel \le \underset{0<\lambda \le {\lambda }_0}{sup}|1+\tau {(1+\tau \lambda )}^{-1}|,$$

for some positive number ${\lambda }_0$. We observe from Figure 1 that the conditions (19) and (20) hold for $\tau >0$.

Figure 1. Plots of (a) ${z=|1-\tau (1+\tau \lambda )}^{-1}|$ and (b) ${z=|1+\tau (1+\tau \lambda )}^{-1}|$ for $0<\lambda \le 1$ and $0<\tau \le 500$.

Finally, we need a technical result for the next two sections.

Lemma 2.

Let Assumptions 1 and 2 hold for the operator $S:=F^{\prime }\left(w^{+}\right)$. Then, there exists a positive number $c_R$ such that

$$i)\parallel I-{\tau }_k{R}_{w_k^{\epsilon }}^{*}f(-{\tau }_k{S}_k{S}_k^{*})\parallel \le c_R\parallel w^{+}-w_k^{\epsilon }\parallel$$

(21)

and

$$ii)\parallel (1-{\alpha }_k)I-{\tau }_k{R}_{w_k^{\epsilon }}^{*}f(-{\tau }_k{S}_k{S}_k^{*})\parallel \le c_R\parallel w^{+}-w_k^{\epsilon }\parallel .$$

(22)

Proof. 

(i)  Following the proof technique of Theorem 1 in [22] and using (17), we have

$$1\le {\widehat{c}}_R^{-1}\parallel R_w-I\parallel$$

(23)

and

$$\parallel I+R_w^{*}\parallel \le {\widehat{c}}_R^{-1}\parallel I-R_w^{*}\parallel \parallel I+R_w^{*}\parallel .$$

(24)

Using Assumption 2; the estimates in Equations (15), (16), (19), (20), (24); and the triangle inequality, we obtain

$$\begin{array}{cc}\hfill & \parallel I-{\tau }_k{R}_{w_k^{\epsilon }}^{*}f(-{\tau }_k{S}_k{S}_k^{*})\parallel \hfill \\ \hfill =& ∥\frac{1}{2}\left[(I+R_{w_k^{\epsilon }}^{*})(I-{\tau }_{k}f(-{\tau }_k{S}_k{S}_k^{*}))\right]+\frac{1}{2}\left[(I-R_{w_k^{\epsilon }}^{*})(I+{\tau }_{k}f(-{\tau }_k{S}_k{S}_k^{*}))\right]∥\hfill \\ \hfill \le & \frac{1}{2}\parallel I+R_{w_k^{\epsilon }}^{*}\parallel \parallel I-{\tau }_{k}f(-{\tau }_k{S}_k{S}_k^{*})\parallel +\frac{1}{2}\parallel I-R_{w_k^{\epsilon }}^{*}\parallel \parallel I+{\tau }_{k}f(-{\tau }_k{S}_k{S}_k^{*})\parallel \hfill \\ \hfill \le & \frac{1}{2}\left[{\widehat{c}}_R^{-1}{D}_1\parallel I+R_{w_k^{\epsilon }}^{*}\parallel +D_2\right]\parallel I-R_{w_k^{\epsilon }}^{*}\parallel \hfill \\ \hfill \le & c_R\parallel w^{+}-w_k^{\epsilon }\parallel ,\hfill \end{array}$$

(25)

with positive number $c_R=\frac{1}{2}\left[{\widehat{c}}_R^{-1}{D}_1(\parallel I\parallel +c_r)+D_2\right]c_L$.

(ii)  Denote $A_k={\tau }_k{R}_{w_k^{\epsilon }}^{*}f(-{\tau }_k{S}_k{S}_k^{*})$. We have

$$\begin{array}{ccc}\hfill \parallel (1-{\alpha }_k)I-A_k\parallel & \le & \frac{1}{2}∥[1-(1+{\alpha }_k)](I+A_k)+[1+(1-{\alpha }_k)](I-A_k)∥\hfill \\ & \le & \frac{|{\alpha }_k|}{2}\parallel I+A_k\parallel +\frac{|2-{\alpha }_k|}{2}\parallel I-A_k\parallel .\hfill \end{array}$$

(26)

Part (i) ensures an upper bound for the second term of the last formula. Hence, a similar upper bound for $\parallel I+A_k\parallel$ remains to be determined. To this end, we will use the inequality $I+PQ=\frac{1}{2}[(I-P)(I-Q)+(I+P)(I+Q)]$ applied to $P=R_{w_k^{\epsilon }}^{*}$ and $Q={\tau }_{k}f(-{\tau }_k{S}_k{S}_k^{*})$. Thus, by using (19) and (20), we obtain

$$\begin{array}{ccc}\hfill \parallel I+A_k\parallel & \le & \frac{1}{2}[(I-R_{w_k^{\epsilon }}^{*})(I-{\tau }_{k}f(-{\tau }_k{S}_k{S}_k^{*}))+(I+R_{w_k^{\epsilon }}^{*})(I+{\tau }_{k}f(-{\tau }_k{S}_k{S}_k^{*}))]\hfill \\ & \le & \frac{D_1}{2}\parallel I-R_{w_k^{\epsilon }}^{*}\parallel +\frac{D_2}{2}\parallel I+R_{w_k^{\epsilon }}^{*}\parallel \hfill \\ & \le & c\parallel w^{+}-w_k^{\epsilon }\parallel ,\hfill \end{array}$$

for some positive c, where the last inequality follows as in (24) and (25). Now, (26) combined with part (i) and the last inequality yield (22). □

3. Convergence Rate for the Iterative RK-Type Regularization

To investigate the convergence rate of the RK-type regularization method (5) under the logarithmic source condition, the nonlinear operator F has to satisfy the local property in an open ball $B_{\rho }\left(w_0\right)$ of radius $\rho$ around $w_0$

$$\begin{array}{c}\hfill ∥F\left(w\right)-F\left(\tilde{w}\right)-F^{\prime }\left(w\right)(w-\tilde{w})∥\le \eta ∥F\left(w\right)-F\left(\tilde{w}\right)∥,\eta <\frac{1}{2},\end{array}$$

(27)

with $w,\tilde{w}$∈$B_{\rho }\left(w_0\right)$$\subset \mathcal{D}\left(F\right)$. In addition, the regularization parameter $k_{*}$ is chosen according to the generalized discrepancy principle, i.e., the iteration is stopped after $k_{*}$ steps with

$$\begin{array}{c}\hfill ∥g^{\epsilon }-F\left(w_{k_{*}}^{\epsilon }\right)∥\le \gamma \epsilon <∥g^{\epsilon }-F\left(w_k^{\epsilon }\right)∥,0\le k<k_{*},\end{array}$$

(28)

where $\gamma >\frac{2-\eta }{1-\eta }$ is a positive number. Note that the triangle inequality yields

$$\begin{array}{c}\hfill \frac{1}{1+\eta }∥F^{\prime }\left(w\right)(w-\tilde{w})∥\le ∥F\left(w\right)-F\left(\tilde{w}\right)∥\le \frac{1}{1-\eta }∥F^{\prime }\left(w\right)(w-\tilde{w})∥.\end{array}$$

(29)

In the sequel, we establish an error estimate that will be useful in deriving the logarithmic convergence rate.

Theorem 1.

Let Assumptions 1 and 2 be valid. Assume that problem (1) has a solution $w^{+}$ in $B_{\frac{\rho }{2}}\left(w_0\right)$ and $g^{\epsilon }$ fulfills (2). Furthermore, assume that the Fr$\acute{e}$chet derivative of F is scaled such that $∥F^{\prime }\left(w\right)∥\le 1$ for $w\in B_{\frac{\rho }{2}}\left(w_0\right)$ and the source conditions (8) and (9) are fulfilled. Thereby, the iterative RK-type regularization is stopped according to the discrepancy principle (28). If $∥v∥$ is sufficiently small, then there exists a constant c depending only on p and $∥v∥$ such that for any $0\le k<k_{*}$,

$$\begin{array}{c}\hfill ∥w^{+}-w_k^{\epsilon }∥\le c{(lnk)}^{-p}\end{array}$$

(30)

and

$$\begin{array}{c}\hfill ∥g^{\epsilon }-F\left(w_k^{\epsilon }\right)∥\le 4c{(k+1)}^{-1/2}{(lnk)}^{-p}.\end{array}$$

Proof. 

Using (5), we can show that

$$\begin{array}{cc}\hfill e_{k+1}=& w^{+}-w_k^{\epsilon }+{\tau }_{k}f(-{\tau }_k{S}_k^{*}{S}_k)F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(F\left(w_k^{\epsilon }\right)-g^{\epsilon })\hfill \\ \hfill =& (I-S^{*}S)e_k+S^{*}S{e}_k+{\tau }_{k}f(-{\tau }_k{S}_k^{*}{S}_k)F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(F\left(w_k^{\epsilon }\right)-g^{\epsilon })\hfill \\ \hfill =& (I-S^{*}S)e_k+S^{*}[F\left(w_k^{\epsilon }\right)-F\left(w^{+}\right)-S(w_k^{\epsilon }-w^{+})]\hfill \\ \hfill & +S^{*}[F\left(w^{+}\right)-g^{\epsilon }+g^{\epsilon }-F\left(w_k^{\epsilon }\right)]+{\tau }_{k}f(-{\tau }_k{S}_k^{*}{S}_k)F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(F\left(w_k^{\epsilon }\right)-g^{\epsilon })\hfill \\ \hfill =& (I-S^{*}S)e_k+S^{*}[F\left(w_k^{\epsilon }\right)-F\left(w^{+}\right)-S(w_k^{\epsilon }-w^{+})]+S^{*}(g-g^{\epsilon })\hfill \\ \hfill & +S^{*}[g^{\epsilon }-F\left(w_k^{\epsilon }\right)]+{\tau }_{k}f(-{\tau }_k{S}_k^{*}{S}_k)F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(F\left(w_k^{\epsilon }\right)-g^{\epsilon })\hfill \\ \hfill =& (I-S^{*}S)e_k+S^{*}[F\left(w_k^{\epsilon }\right)-F\left(w^{+}\right)-S(w_k^{\epsilon }-w^{+})]+S^{*}(g-g^{\epsilon })\hfill \\ \hfill & +[S^{*}-{\tau }_{k}f(-{\tau }_k{S}_k^{*}{S}_k)F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}](g^{\epsilon }-F\left(w_k^{\epsilon }\right)).\hfill \end{array}$$

(31)

Using the spectral theory and (14), we have

$$f(-{\tau }_k{S}_k^{*}{S}_k)F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}=F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}f(-{\tau }_k{S}_k{S}_k^{*})=S^{*}{R}_{w_k^{\epsilon }}^{*}f(-{\tau }_k{S}_k{S}_k^{*}).$$

(32)

Consequently, Equation (31) can be rewritten as

$$\begin{array}{cc}\hfill e_{k+1}=& (I-S^{*}S)e_k+S^{*}[F\left(w_k^{\epsilon }\right)-F\left(w^{+}\right)-S(w_k^{\epsilon }-w^{+})]+S^{*}(g-g^{\epsilon })\hfill \\ \hfill & +S^{*}[I-{\tau }_k{R}_{w_k^{\epsilon }}^{*}f(-{\tau }_k{S}_k{S}_k^{*})](g^{\epsilon }-F\left(w_k^{\epsilon }\right))\hfill \\ \hfill =& (I-S^{*}S)e_k+S^{*}(g-g^{\epsilon })+S^{*}{z}_k,\hfill \end{array}$$

(33)

where

$$z_k=F\left(w_k^{\epsilon }\right)-F\left(w^{+}\right)-S(w_k^{\epsilon }-w^{+})+[I-{\tau }_k{R}_{w_k^{\epsilon }}^{*}f(-{\tau }_k{S}_k{S}_k^{*})](g^{\epsilon }-F\left(w_k^{\epsilon }\right)).$$

(34)

By recurrence and Equation (33), we obtain

$$\begin{array}{c}\hfill e_k={(I-S^{*}S)}^k{e}_0+\sum _{j=1}^k{(I-S^{*}S)}^{j-1}{S}^{*}(g-g^{\epsilon })+\sum _{j=1}^k{(I-S^{*}S)}^{j-1}{S}^{*}{z}_{k-j}.\end{array}$$

(35)

Moreover, it holds that

$$\begin{array}{c}\hfill S{e}_k={(I-S{S}^{*})}^{k}S{e}_0+\sum _{j=1}^k{(I-S{S}^{*})}^{j-1}S{S}^{*}(g-g^{\epsilon })+\sum _{j=1}^k{(I-S{S}^{*})}^{j-1}S{S}^{*}{z}_{k-j}.\end{array}$$

(36)

We will prove by mathematical induction that

$$\parallel e_k{\parallel \le c(ln(k+e))}^{-p}$$

(37)

and

$$\parallel S{e}_k{\parallel \le c(k+1)}^{-1/2}{(ln(k+e))}^{-p}$$

(38)

hold for all $0\le k<k_{*}$ with a positive constant c independent of k. Using the discrepancy principle (28), triangle inequality, and $\gamma >{\displaystyle \frac{2-\eta }{1-\eta }}$, we can show that

$$\parallel g^{\epsilon }-F\left(w_k^{\epsilon }\right)\parallel \le 2\parallel g^{\epsilon }-F\left(w_k^{\epsilon }\right)\parallel -\gamma \epsilon \le 2\parallel g-F\left(w_k^{\epsilon }\right)\parallel \le \frac{2}{1-\eta }\parallel S{e}_k\parallel .$$

(39)

Using Proposition 1, Lemma 2, (34), and (39), it follows that

$$\begin{array}{cc}\hfill \parallel z_k\parallel \le & \parallel F\left(w_k^{\epsilon }\right)-F\left(w^{+}\right)-S(w_k^{\epsilon }-w^{+})\parallel +\parallel I-{\tau }_k{R}_{w_k^{\epsilon }}^{*}f(-{\tau }_k{S}_k{S}_k^{*})\parallel \parallel g^{\epsilon }-F\left(w_k^{\epsilon }\right)\parallel \hfill \\ \hfill \le & \frac{1}{2}{c}_L\parallel e_k\parallel \parallel S{e}_k\parallel +\frac{2c_R}{1-\eta }\parallel e_k\parallel \parallel S{e}_k\parallel \hfill \\ \hfill \le & {\widehat{c}}_1\parallel e_k\parallel \parallel S{e}_k\parallel ,\hfill \end{array}$$

(40)

with ${\widehat{c}}_1\ge \frac{1}{2}{c}_L+\frac{2c_R}{1-\eta }$.

By assumption $\parallel S\parallel \le 1$ (see Vainikko and Veterennikov [23], as cited in Hanke et al. [5]), we have

$$∥\sum _{j=0}^{k-1}{(I-S^{*}S)}^j{S}^{*}∥\le \sqrt{k}$$

(41)

and

$$∥{(I-S^{*}S)}^j{S}^{*}∥\le {(j+1)}^{-1/2},j\ge 1.$$

(42)

Therefore,

$$\begin{array}{cc}\hfill ∥\sum _{j=1}^k{(I-S^{*}S)}^{j-1}{S}^{*}(g-g^{\epsilon })∥=& ∥\sum _{j=0}^{k-1}{(I-S^{*}S)}^j{S}^{*}(g-g^{\epsilon })∥\hfill \\ \hfill \le & \sqrt{k}\epsilon \hfill \end{array}$$

(43)

and

$$\begin{array}{cc}\hfill ∥\sum _{j=1}^k{(I-S^{*}S)}^{j-1}{S}^{*}{z}_{k-j}∥=& ∥\sum _{j=0}^{k-1}{(I-S^{*}S)}^j{S}^{*}{z}_{k-j-1}∥\hfill \\ \hfill \le & \sum _{j=0}^{k-1}{(j+1)}^{-1/2}\parallel z_{k-j-1}\parallel .\hfill \end{array}$$

(44)

Using Lemma 1, (40), (43), and (44), Equation (35) becomes

$$\begin{array}{cc}\hfill \parallel e_k\parallel \le & c_1{(ln(k+e))}^{-p}\parallel v\parallel +\sqrt{k}\epsilon +\sum _{j=0}^{k-1}{(j+1)}^{-1/2}\parallel z_{k-j-1}\parallel \hfill \\ \hfill \le & c_1{(ln(k+e))}^{-p}\parallel v\parallel +\sqrt{k}\epsilon +{\widehat{c}}_1\sum _{j=0}^{k-1}{(j+1)}^{-1/2}\parallel e_{k-j-1}\parallel \parallel S{e}_{k-j-1}\parallel .\hfill \end{array}$$

(45)

Employing the assumption of the induction in Equations (37) and (38) into the third term of (45), we obtain

$$\begin{array}{cc}\hfill & \sum _{j=0}^{k-1}{(j+1)}^{-1/2}\parallel e_{k-j-1}\parallel \parallel S{e}_{k-j-1}\parallel \hfill \\ \hfill \le & c^2\sum _{j=0}^{k-1}{(j+1)}^{-1/2}{(k-j)}^{-1/2}{(ln(k-j-1+e))}^{-2p}\hfill \\ \hfill =& c^2\sum _{j=0}^{k-1}{\left(\frac{j+1}{k+1}\right)}^{-1/2}{\left(\frac{k-j}{k+1}\right)}^{-1/2}{(ln(k-j-1+e))}^{-2p}\left(\frac{1}{k+1}\right)\hfill \\ \hfill \le & c^2{(ln(k+e))}^{-p}\sum _{j=0}^{k-1}{\left(\frac{j+1}{k+1}\right)}^{-1/2}{\left(\frac{k-j}{k+1}\right)}^{-1/2}\left(\frac{1}{k+1}\right){\left[\frac{ln(k+e)}{ln(k-j-1+e)}\right]}^{2p}.\hfill \end{array}$$

(46)

Similar to Equation (45) in [22], we have

$$\begin{array}{cc}\hfill \frac{ln(k+e)}{ln(k-j-1+e)}\le & E\left(1+ln\left(\frac{k+1}{k-j-1+e}\right)\right)\hfill \end{array}$$

(47)

with a generic constant $E<2$, which does not depend on $k\ge 1$. Using (47), we can estimate (46) as

$$\begin{array}{cc}\hfill & \sum _{j=0}^{k-1}{(j+1)}^{-1/2}\parallel e_{k-j-1}\parallel \parallel S{e}_{k-j-1}\parallel \hfill \\ \hfill \le & c^2{E}^{2p}{(ln(k+e))}^{-p}\sum _{j=0}^{k-1}{\left(\frac{j+1}{k+1}\right)}^{-1/2}{\left(\frac{k-j}{k+1}\right)}^{-1/2}\left(\frac{1}{k+1}\right){\left(1+ln\left(\frac{k+1}{k-j-1+e}\right)\right)}^{2p}\hfill \\ \hfill \le & c^2{E}^{2p}{(ln(k+e))}^{-p}\sum _{j=0}^{k-1}{\left(\frac{j+1}{k+1}\right)}^{-1/2}{\left(\frac{k-j}{k+1}\right)}^{-1/2}\left(\frac{1}{k+1}\right){\left(1-ln\left(\frac{k-j}{k+1}\right)\right)}^{2p}.\hfill \end{array}$$

(48)

The sum ${\sum }_{j=0}^{k-1}·$ is bounded because the integral

$${\int }_s^{1-s}{x}^{-1/2}{(1-x)}^{-1/2}{(1-ln(1-x))}^{2p}dx$$

is bounded with $s:=\frac{1}{2(k+1)}$ from above by a positive constant $E_p$ independent of k. Thus, (45) becomes

$$\begin{array}{cc}\hfill \parallel e_k\parallel \le & c_1{(ln(k+e))}^{-p}\parallel v\parallel +\sqrt{k}\epsilon +{\widehat{c}}_1{c}^2{E}^{2p}{E}_p{(ln(k+e))}^{-p}\hfill \\ \hfill =& \left[c_1\parallel v\parallel +c_p{c}^2\right]{(ln(k+e))}^{-p}+\sqrt{k}\epsilon ,\hfill \end{array}$$

(49)

with $c_p={\widehat{c}}_1{E}^{2p}{E}_p$.

By assumption $\parallel S\parallel \le 1$ (see Vainikko and Veterennikov [23] as cited in Hanke et al. [5]), we have

$$∥\sum _{j=0}^{k-1}{(I-S{S}^{*})}^{j}S{S}^{*}∥\le \parallel I-{(I-S{S}^{*})}^k\parallel \le 1$$

(50)

and

$$\parallel {(I-S{S}^{*})}^{j}S{S}^{*}\parallel \le {(j+1)}^{-1}.$$

(51)

Thus,

$$\begin{array}{cc}\hfill ∥\sum _{j=1}^k{(I-S{S}^{*})}^{j-1}S{S}^{*}(g-g^{\epsilon })∥=& ∥\sum _{j=0}^{k-1}{(I-S{S}^{*})}^{j}S{S}^{*}(g-g^{\epsilon })∥\le \epsilon \hfill \end{array}$$

(52)

and

$$\begin{array}{cc}\hfill ∥\sum _{j=1}^k{(I-S{S}^{*})}^{j-1}S{S}^{*}{z}_{k-j}∥=& ∥\sum _{j=0}^{k-1}{(I-S{S}^{*})}^{j}S{S}^{*}{z}_{k-j-1}∥\hfill \\ \hfill \le & \sum _{j=0}^{k-1}{(j+1)}^{-1}\parallel z_{k-j-1}\parallel .\hfill \end{array}$$

(53)

Using Lemma 1, (40), (52), and (53), Equation (36) can be estimated as

$$\begin{array}{cc}\hfill \parallel S{e}_k\parallel \le & c_2{(k+1)}^{-1/2}{(ln(k+e))}^{-p}\parallel v\parallel +\epsilon +{\widehat{c}}_1\sum _{j=0}^{k-1}{(j+1)}^{-1}\parallel e_{k-j-1}\parallel \parallel S{e}_{k-j-1}\parallel .\hfill \end{array}$$

(54)

Using (47) and the assumption of the induction in Equations (37) and (38) into the third term of (54), we obtain

$$\begin{array}{cc}\hfill & \sum _{j=0}^{k-1}{(j+1)}^{-1}\parallel e_{k-j-1}\parallel \parallel S{e}_{k-j-1}\parallel \hfill \\ \hfill \le & c^2\sum _{j=0}^{k-1}{(j+1)}^{-1}{(k-j)}^{-1/2}{(ln(k-j-1+e))}^{-2p}\hfill \\ \hfill =& c^2{(k+1)}^{-1/2}{(ln(k+e))}^{-p}\hfill \\ \hfill & \times \sum _{j=0}^{k-1}{\left(\frac{j+1}{k+1}\right)}^{-1}{\left(\frac{k-j}{k+1}\right)}^{-1/2}{\left(\frac{ln(k+e)}{ln(k-j-1+e)}\right)}^{2p}{(ln(k+e))}^{-p}\frac{1}{k+1}\hfill \\ \hfill =& c^2{E}^{2p}{(k+1)}^{-1/2}{(ln(k+e))}^{-p}\hfill \\ \hfill & \times \sum _{j=0}^{k-1}{\left(\frac{j+1}{k+1}\right)}^{-1}{\left(\frac{k-j}{k+1}\right)}^{-1/2}{\left(1-ln\left(\frac{k-j}{k+1}\right)\right)}^{2p}\frac{1}{k+1}.\hfill \end{array}$$

(55)

The summation in (55) is bounded because, with $s:=\frac{1}{2(k+1)}$, the integral

$${\int }_s^{1-s}{x}^{-1}{(1-x)}^{-1/2}{(1-ln(1-x))}^{2p}dx\le {\tilde{E}}_p,$$

for some positive constant ${\tilde{E}}_p$ independently of k. Thus, (54) becomes

$$\begin{array}{cc}\hfill \parallel S{e}_k\parallel \le & c_2{(k+1)}^{-1/2}{(ln(k+e))}^{-p}\parallel v\parallel +\epsilon +{\widehat{c}}_1{c}^2{E}^{2p}{\tilde{E}}_p{(k+1)}^{-1/2}{(ln(k+e))}^{-p}\hfill \\ \hfill =& [c_2\parallel v\parallel +{\tilde{c}}_p{c}^2{](k+1)}^{-1/2}{(ln(k+e))}^{-p}+\epsilon \hfill \end{array}$$

(56)

with ${\tilde{c}}_p={\widehat{c}}_1{E}^{2p}{\tilde{E}}_p$. Setting $c_{*}=max\{c_1,c_2\}$, we have

$$\parallel e_k\parallel \le [c_{*}\parallel v\parallel +c_p{c}^2{](ln(k+e))}^{-p}+\sqrt{k}\epsilon$$

(57)

and

$$\parallel S{e}_k\parallel \le [c_{*}\parallel v\parallel +{\tilde{c}}_p{c}^2{](k+1)}^{-1/2}{(ln(k+e))}^{-p}+\epsilon .$$

(58)

The discrepancy principles (28) and (29) provide

$$\gamma \epsilon \le \parallel g^{\epsilon }-F\left(w_k^{\epsilon }\right)\parallel \le \epsilon +\frac{1}{1-\eta }\parallel S{e}_k\parallel ,0\le k<k_{*}.$$

Using (58), we obtain

$$(1-\eta )(\gamma -1)\epsilon \le \parallel S{e}_k\parallel \le [c_{*}\parallel v\parallel +{\tilde{c}}_p{c}^2{](k+1)}^{-1/2}{(ln(k+e))}^{-p}+\epsilon ,0\le k<k_{*}.$$

(59)

Setting $\omega =(1-\eta )(\gamma -1)-1>0$, (59) leads to

$$\epsilon \le \frac{1}{\omega }\left[c_{*}\parallel v\parallel +{\tilde{c}}_p{c}^2\right]{(k+1)}^{-1/2}{(ln(k+e))}^{-p},0\le k<k_{*}.$$

(60)

Applying (60) to (57), we obtain

$$\parallel e_k\parallel \le \left(1+\frac{1}{\omega }\right)\left[c_{*}\parallel v\parallel +{\widehat{c}}_p{c}^2\right]{(ln(k+e))}^{-p},0\le k<k_{*}$$

(61)

with ${\widehat{c}}_p=max\{c_p,{\tilde{c}}_p\}$.

Applying (60) to (58), we obtain

$$\parallel S{e}_k\parallel \le \left(1+\frac{1}{\omega }\right)\left[c_{*}\parallel v\parallel +{\widehat{c}}_p{c}^2\right]{(k+1)}^{-1/2}{(ln(k+e))}^{-p},0\le k<k_{*}.$$

(62)

We choose a sufficiently small $\parallel v\parallel$ such that $\left(1+\frac{1}{\omega }\right)\left[c_{*}\parallel v\parallel +{\widehat{c}}_p{c}^2\right]\le c$. Thus, the induction is completed. Using (37), we can show that

$$\parallel e_k\parallel \le c{\left(\frac{lnk}{ln(k+e)}\right)}^p{(lnk)}^{-p}\le c{(lnk)}^{-p}.$$

(63)

The second assertion is obtained by using (39) as follows:

$$\begin{array}{cc}\hfill \parallel g^{\epsilon }-F\left(w_k^{\epsilon }\right)\parallel & \le \frac{2}{1-\eta }c{(k+1)}^{-1/2}{\left(\frac{lnk}{ln(k+e)}\right)}^p{(lnk)}^{-p}\hfill \\ \hfill & \le 4c{(k+1)}^{-1/2}{(lnk)}^{-p}.\hfill \end{array}$$

(64)

□

We are now in a position to show the logarithmic convergence rate for the iterative RK-type regularization under a logarithmic source condition, when the iteration is stopped according to the discrepancy principle (28).

Theorem 2.

Under the assumptions of Theorem 1 and for $1\le p\le 2$, one has

$$k_{*}^{\frac{1}{2}}{(ln\left(k_{*}\right))}^p=O(1/\epsilon ),$$

(65)

$$\parallel e_{k_{*}}\parallel =O\left({(-ln\epsilon )}^{-p}\right).$$

(66)

Proof. 

From (60), it follows that

$$\epsilon \le c{(k+1)}^{-1/2}{(ln(k+e))}^{-p}\le c{(k+1)}^{-1/2}{(ln(k+1))}^{-p},0\le k<k_{*},$$

(67)

for some positive constant c. By taking $k_{*}=k-1$, one obtains (65). Furthermore, Lemma (A4) in [22] applied to (65) yields $k_{*}=O\left(\frac{{(-ln\epsilon )}^{-2p}}{{\epsilon }^2}\right)$. For showing the second inequality, we use $e_0=\phi \left(S^{*}S\right)v$ in (30) and proceed as in the proof of Theorem 2 in [22]. □

4. Convergence Rate for the Modified Version of the Iterative RK-Type Regularization

The paper [13] contains a study of the modified iterative Runge-Kutta regularization method

$$w_{k+1}^{\epsilon }=w_k^{\epsilon }+{\tau }_k{b}^T{\Pi }^{-1}𝟙F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(g^{\epsilon }-F\left(w_k^{\epsilon }\right))-{\tau }_k^{-1}(w_k^{\epsilon }-\zeta ),$$

(68)

where $\zeta \in D\left(F\right)$ and $\Pi =\delta +{\tau }_{k}A{F}^{\prime }{\left(w_k^{\epsilon }\right)}^{*}{F}^{\prime }\left(w_k^{\epsilon }\right)$. More precisely, it presents a detailed convergence analysis and derives Hölder-type convergence rates.

The aim in this section is to show convergence rates for (68) with the natural choice $\zeta =w_0$. We consider here the logarithmic source condition (9) with $\phi$ defined by (8), where $\parallel v\parallel$ is small enough. That is, we deal with the following method:

$$w_{k+1}^{\epsilon }=w_k^{\epsilon }+{\tau }_k{b}^T{\Pi }^{-1}𝟙F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(g^{\epsilon }-F\left(w_k^{\epsilon }\right))-{\tau }_k^{-1}(w_k^{\epsilon }-w_0).$$

(69)

We work further under assumptions (27) and (28) with an appropriately chosen constant $\gamma$—compare inequality (2.10) in [13]. For the sake of completeness, we recall below the convergence result adapted to the choice $\zeta =w_0$ (compare to Proposition 2.1 and Theorem 2.1 in [13]).

Theorem 3.

Let $w^{+}$ be a solution of (1) in $B_{\frac{\rho }{8}}\left(w_0\right)$ with $w_0=w_0^{\epsilon }$ and assume that $g^{\epsilon }$ fulfills (2). If the parameters ${\displaystyle {\alpha }_k,\forall k\in {\mathbb{N}}_0}$ with ${\displaystyle \sum _{k=0}^{\infty }{\alpha }_k<\infty }$ are small enough and if the termination index is defined by (28), then $w_{k_{*}}^{\epsilon }\to w^{†}$ as $\epsilon \to 0$.

We state below a result on essential upper bounds for the errors in (69).

Theorem 4.

Let Assumptions 1 and 2 hold for the operator $S:=F^{\prime }\left(w^{†}\right)$. Assume that problem (1) has a solution $w^{+}$ in $B_{\frac{\rho }{8}}\left(w_0\right)$ and $g^{\epsilon }$ fulfills (2). Assume that the Fr$\acute{e}$chet derivative of F is scaled such that $∥F^{\prime }\left(w\right)∥\le 1$ for $w\in B_{\frac{\rho }{8}}\left(w_0\right)$ and that the parameters ${\displaystyle {\alpha }_k,\forall k\in {\mathbb{N}}_0}$ with ${\displaystyle \sum _{k=0}^{\infty }{\alpha }_k<\infty }$ are small enough. Furthermore, assume that the source condition 9 is fulfilled and that the modified RK-type regularization method (69) is stopped according to (28). If $∥v∥$ is sufficiently small, then there exists a constant c depending only on p and $∥v∥$ such that for any $0\le k<k_{*}$,

$$\begin{array}{c}\hfill ∥w^{+}-w_k^{\epsilon }∥\le c{(lnk)}^{-p}\end{array}$$

(70)

and

$$\begin{array}{c}\hfill ∥g^{\epsilon }-F\left(w_k^{\epsilon }\right)∥\le 4c{(k+1)}^{-1/2}{(lnk)}^{-p}.\end{array}$$

(71)

Proof. 

First, we deduce an explicit formula for $e_k=w^{+}-w_k^{\epsilon }$. We proceed similarly to the proof in Theorem 1, but this time, we need to take into account the additional term $-{\alpha }_k(w_k^{\epsilon }-w_0)$. Thus, the proof steps are as follows.

I. We establish an explicit formula for the error $e_k$:

$$\begin{array}{cc}\hfill e_{k+1}=& e_k+{\tau }_{k}f(-{\tau }_k{S}_k^{*}{S}_k)F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(F\left(w_k^{\epsilon }\right)-g^{\epsilon })+{\alpha }_k(w_k^{\epsilon }-w_0)\hfill \\ \hfill =& (1-{\alpha }_k)e_k+{\tau }_{k}f(-{\tau }_k{S}_k^{*}{S}_k)F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(F\left(w_k^{\epsilon }\right)-g^{\epsilon })+{\alpha }_k(w^{+}-w_0)\hfill \\ \hfill =& (1-{\alpha }_k)(I-S^{*}S)e_k+(1-{\alpha }_k)S^{*}[F\left(w_k^{\epsilon }\right)-F\left(w^{+}\right)-S(w_k^{\epsilon }-w^{+})]\hfill \\ \hfill +& (1-{\alpha }_k)S^{*}(g-g^{\epsilon })+(1-{\alpha }_k)S^{*}(g^{\epsilon }-F\left(w_k^{\epsilon }\right))+{\alpha }_k(w^{+}-w_0)\hfill \\ \hfill & -{\tau }_{k}f(-{\tau }_k{S}_k^{*}{S}_k)F^{\prime }{\left(w_k^{\epsilon }\right)}^{*}(g^{\epsilon }-F\left(w_k^{\epsilon }\right))\hfill \\ \hfill =& (1-{\alpha }_k)(I-S^{*}S)e_k+(1-{\alpha }_k)S^{*}{q}_k\hfill \\ \hfill +& (1-{\alpha }_k)S^{*}(g-g^{\epsilon })+S^{*}(I-{\tau }_k{R}_{w_k^{\epsilon }}^{*}f(-{\tau }_k{S}_k{S}_k^{*}))(g^{\epsilon }-F\left(w_k^{\epsilon }\right))\hfill \\ \hfill +& {\alpha }_k{S}^{*}\left(F(w_k^{\epsilon }-g^{\epsilon })\right)+{\alpha }_k(w^{+}-w_0),\hfill \end{array}$$

(72)

where the last equality follows from (32). We denoted $q_k=F\left(w_k^{\epsilon }\right)-F\left(w^{+}\right)-S(w_k^{\epsilon }-w^{+})$.

Thus, (72) can be shortly rewritten as

$$e_{k+1}=(1-{\alpha }_k)(I-S^{*}S)e_k+(1-{\alpha }_k)S^{*}(g-g^{\epsilon })+S^{*}{z}_k+{\alpha }_k(w^{+}-w_0)$$

(73)

with

$$z_k=(1-{\alpha }_k)q_k+[(1-{\alpha }_k)I-{\tau }_k{R}_{w_k^{\epsilon }}^{*}f(-{\tau }_k{S}_k{S}_k^{*})](g^{\epsilon }-F\left(w_k^{\epsilon }\right)).$$

(74)

Therefore, we obtain the following closed formula:

$$\begin{array}{cc}\hfill e_k=& \left[\prod _{j=0}^{k-1}(1-{\alpha }_j){(I-S^{*}S)}^k+\sum _{j=0}^{k-1}{\alpha }_{k-j-1}{(I-S^{*}S)}^j\prod _{l=1}^j(1-{\alpha }_{k-l})\right]e_0\hfill \\ \hfill +& \left[\sum _{j=1}^k{(I-S^{*}S)}^{j-1}\prod _{l=1}^j(1-{\alpha }_{k-l})\right]S^{*}(y-y^{\epsilon })\hfill \\ \hfill +& \sum _{j=0}^{k-1}\prod _{l=k-j}^{k-1}(1-{\alpha }_l){(I-S^{*}S)}^j{S}^{*}{z}_{k-j-1}.\hfill \end{array}$$

(75)

From Proposition 1, (34), Lemma 2, (39), and (74), it follows that

$$\parallel z_k\parallel \le \widehat{c}\parallel e_k\parallel \parallel S{e}_k\parallel ,$$

for any $0\le k\le k_{*}$, where $\widehat{c}$ is a positive constant.

II. Following the technical steps of the proof of Theorem 1 in [22], one can similarly show by induction that there is a positive number c, such that the following inequalities hold for any $0\le k\le k_{*}$:

$$\parallel e_k{\parallel \le c(ln(k+e))}^{-p},$$

(76)

$$\parallel S{e}_k{\parallel \le c(k+1)}^{-1/2}{(ln(k+e))}^{-p}.$$

(77)

Then, one can eventually obtain (70) and (71) as in the mentioned proof. Note that Theorem 1 in [22] is based on Proposition 1 in [22], which requires small enough parameters ${\alpha }_k$, such that ${\displaystyle \sum _{k=0}^{\infty }{\alpha }_k<\infty }$ and ${\alpha }_{n-k+1}\le {\left(\frac{ln(k+e)}{ln(k+1+e)}\right)}^p$, for all $n>k\ge 1$ (compare to (16) on page 4 in [22]). Since the smallest value of $\frac{ln(k+e)}{ln(k+1+e)}$ is about $0.84$ (when $k=1$), one can clearly find ${\alpha }_k\in (0,1)$ small enough so as to satisfy the imposed inequalities, e.g., a harmonic-type sequence such as ${\alpha }_k=1/{(k+2)}^r$ for some $r>1$. □

One can show convergence rates for the modified Runge-Kutta regularization method, as done in the previous section for the unmodified version.

Theorem 5.

Under the assumptions of Theorem 4 and for $1\le p\le 2$, one has

$$k_{*}^{\frac{1}{2}}{(ln\left(k_{*}\right))}^p=O(1/\epsilon ),$$

$$\parallel e_{k_{*}}\parallel =O\left({(-ln\epsilon )}^{-p}\right).$$

5. Numerical Example

The purpose of the following numerical example is to verify the error estimates shown above. Define the nonlinear operator $F:L^2[0,1]\to L^2[0,1]$ as

$$\left[F\left(w\right)\right]\left(s\right)=exp{\int }_0^{1}k(s,t)w\left(t\right)dt,$$

(78)

with the kernel function

$$k(s,t)=\left\{\begin{array}{cc}s(1-t)\hfill & \mathrm{if} s<t;\hfill \\ t(1-s)\hfill & \mathrm{if} t\le s.\hfill \end{array}\right.$$

(79)

The noisy data is given by $g^{\epsilon }\left(s\right)=exp(sin\left(\pi s\right)/{\pi }^2)+\epsilon cos\left(100s\right),s\in [0,1],$ and the exact solution is $w_{*}\left(t\right)=sin\left(\pi t\right)$. In order to demonstrate the results in Theorems 1 and 4, we consider Landweber, Levenberg–Marquardt (LM), Lobatto IIIC, and the Radau IIA methods, see Table 1 for the Butcher tableau.

Table 1. Butcher tableau for (a) explicit Euler or Landweber, (b) implicit Euler or Levenberg–Marquardt, (c) Lobatto IIIC, and (d) Radau IIA methods.

0

$\frac{1}{2}$

$-\frac{1}{2}$

$\frac{1}{3}$

$\frac{5}{12}$

$-\frac{1}{12}$

0

0

1

1

1

$\frac{1}{2}$

$\frac{1}{2}$

1

$\frac{3}{4}$

$\frac{1}{4}$

1

1

$\frac{1}{2}$

$\frac{1}{2}$

$\frac{3}{4}$

$\frac{1}{4}$

(a)

(b)

(c)

(d)

The implementation in this section is the same as the one reported in [10,13]. The number of basis functions is 65 and the number of equidistant grid points is 150, while the parameter ${\tau }_k$ is the harmonic sequence term ${(k+1)}^{1.1}$. As expected, the results in Figure 2 show that the curve of $ln\parallel w^{+}-w_k^{\epsilon }\parallel$ lies below a straight line with slope $-p$, as suggested by (30) in Theorem 1 and (70) in Theorem 4.

Figure 2. The plot of $ln\parallel w^{+}-w_k^{\epsilon }\parallel$ versus $ln(ln(k\left)\right)$ for (a) one−step RK methods and (b) for two−step methods with $\epsilon ={10}^{-3}$. (c,d) Results with $\epsilon ={10}^{-4}$. The parameters $\gamma$ are (a) $1.1$, (b) 25, (c) $1.1$, and (d) 100. For (a–d), a harmonic sequence ${\tau }_k={(k+1)}^{1.1}$ and $w_0=\epsilon w_{*}$ are used.

6. Summary and Outlook

Up to now, the logarithmic convergence rate under logarithmic source condition has only been investigated for particular examples, namely, the Levenberg–Marquardt method (Böckmann et al. [21]) and the modified Landweber method (Pornsawad et al. [22]). Here, we extended the results to the whole family of Runge-Kutta-type methods with and without modification. For the future, it is still open to prove the optimal convergence rate under Hölder source condition for the whole family without modification.