Convergence Rate of Runge-Kutta-Type Regularization for Nonlinear Ill-Posed Problems under Logarithmic Source Condition

: We prove the logarithmic convergence rate of the families of usual and modiﬁed iterative Runge-Kutta methods for nonlinear ill-posed problems between Hilbert spaces under the logarithmic source condition, and numerically verify the obtained results. The iterative regularization is terminated by the a posteriori discrepancy principle.


Introduction
Let X and Y be infinite-dimensional real Hilbert spaces with inner products ·, · and norms · . Let us consider a nonlinear ill-posed operator equation where F : D(F) ⊂ X → Y is a nonlinear operator between the Hilbert spaces X and Y. We assume that (1) has a solution w + for exact data (which need not be unique). We have approximate data g ε with g ε − g ≤ ε, ε > 0.
Besides the classical Tikhonov-Phillips regularization, a plethora of interesting variational and iterative approaches for ill-posed problems can be found, e.g., in Morozov [1], Tikhonov and Arsenin [2], Bakushinsky and Kokurin [3], and Kaltenbacher et al. [4]. We focus here on iterative methods, as they are also very popular and effective to use in applications. The simplest iterative regularization is the Landweber method-see, e.g., Hanke et al. [5], where the analysis for convergence rates is done under Hölder-type source condition. A more effective method often used in applications is the Levenberg-Marquardt method This was investigated in [6][7][8] under the Hölder-type source condition (HSC) and a posteriori discrepancy principle (DP). Jin [7] proved optimal convergence rates for an a priori chosen geometric step size sequence α k , whereas Hochbruck and Hönig [6] showed convergence with the optimal rate for quite general step size sequences including the geometric sequence. Later, Hanke [8] avoided any constraints on the rate of decay of the regularization parameter to show the optimal convergence rate.
Tautenhahn [9] proved that asymptotic regularization, i.e., the approximation of problem (1) by a solution of the Showalter differential equation (SDE) where the regularization parameter T is chosen according to the DP under the HSC, is a stable method to solve nonlinear ill-posed problems. Solving SDE by the family of Runge-Kutta (RK) methods delivers a family of RK-type iterative regularization methods where 1 denotes the (s × 1) vector of identity operators, while δ is the (s × s) diagonal matrix of bounded linear operators with identity operator on the entire diagonal and zero operator outside of the main diagonal with respect to the appropriate spaces. The parameter τ k = 1/α k in (5) is the step-length, also called the relaxation parameter. The (s × s) matrix A and the (s × 1) vector b are the given parameters that correspond to the specific RK method, building the so-called Butcher tableau (succession of stages). Different choices of the RK parameters generate various iterative methods. Böckmann and Pornsawad [10] showed convergence for the whole RK-type family (including the well-known Landweber and the Levenberg-Marquardt methods). That paper also emphasized advantages of using some procedures from the mentioned family, e.g., regarding implicit A-stable Butcher tableaux. For instance, the Landweber method needs a lot of iteration steps, while those implicit methods need only a few iteration steps, thus minimizing the rounding errors.
Later, Pornsawad and Böckmann [11] filled in the missing results on optimal convergence rates, but only for particular first-stage methods under HSC and using DP.
Our current study considers further the unifying RK-framework described above, as well as a modified version presented below, showing optimality of the RK-regularization schemes under logarithmic source conditions. An additional term α k (w ε k − ξ), as in the iteratively regularized Gauss-Newton method (see, e.g., [4]), was added to a modified Landweber method. Thus, Scherzer [12] proved a convergence rate result under HSC without particular assumptions on the nonlinearity of operator F. Moreover, in Pornsawad and Böckmann [13], an the additional term was included to the whole family of iterative RK-type methods (which contains the modified Landweber iteration), where ζ ∈ X and Π = δ + τ k AF (w ε k ) * F (w ε k ). Using a priori and a posteriori stopping rules, the convergence rate results of the RK-type family are obtained under HSC if the Fréchet derivative is properly scaled.
Due to the minimal assumptions for the convergence analysis of the modified iterative RK-type methods, an additional term was added to SDE Pornsawad et al. [14] investigated this continuous version of the modified iterative RK-type methods for nonlinear inverse ill-posed problems. The convergence analysis yields the optimal rate of convergence under a modified DP and an exponential source condition.
Recently, a second-order asymptotic regularization for the linear problemÂx = y was investigated in [15]ẍ (t) + µẋ(t) +Â * Â x(t) =Â * y δ , x(0) =x,ẋ(0) =ẋ under HSC using DP. Define with p > 0 and the usual logarithmic sourcewise representation where v is sufficiently small and w 0 ∈ D(F) is an initial guess that may incorporate a priori knowledge on the solution. In numerous applications-e.g., heat conduction, scattering theory, which are severely illposed problems-the Hölder source condition is far too strong. Therefore, Hohage [16] proved convergence and logarithmic convergence rates for the iteratively regularized Gauss-Newton method in a Hilbert space setting, provided a logarithmic source condition (9) is satisfied and DP is used as the stopping rule. Deuflhard et al. [17] showed some convergence rate result for the Landweber iteration using DP and (9) under a Newton-Mysovskii condition on the nonlinear operator. Sufficient conditions for the convergence rate, which is logarithmic in the data noise level, are given.
In Hohage [18], a systematic study of convergence rates for regularization methods under (9) including the case of operator approximations for a priori and a posteriori stopping rules is provided. A logarithmic source condition is considered by Pereverzyev et al. [19] for a derivative-free method, by Mahale and Nair [20] for a simplified generalized Gauss-Newton method, and by Böckmann et al. [21] for the Levenberg-Marquardt method using DP as stopping rule. Pornsawad et al. [22] solved the inverse potential problem, which is exponentially ill-posed, employing the modified Landweber method and proved convergence rate under the logarithmic source condition via DP for this method.
To the best of our knowledge, for the first time, convergence rates are established both for the whole family of RK-methods and for the modified version, when applied to severely ill-posed problems (i.e., under the logarithmic source condition).
The structure of this article is as follows. Section 2 provides assumptions and technical estimations. We derive the convergence rate of the RK-type method (5) in Section 3 and of the modified RK-type method (6) in Section 4 under the logarithmic source conditions (8) and (9). In Section 5, the performed numerical experiments confirm the theoretical results.

Assumption 1.
There exist positive constants c L , c r , and c R and a linear bounded operator R w : Y → Y such that for w ∈ B ρ (w 0 ), the following conditions hold where w + is the exact solution of (1).
Let e k := w + − w ε k be the error of the kth iteration w ε k , S k := F (w ε k ) and S := F (w + ). (14) and (15) in Assumption 1 hold. Then,

Proposition 1. Let the conditions
The proof is given in [22] using the mean value theorem.

Assumption 2.
Let K be a linear operator and τ be a positive number. There exist positive constants D 1 and D 2 such that and We note that the explicit Euler method provides I − τ f (−τKK * ) ≤ |1 − τ| and I + τ f (−τKK * ) ≤ |1 + τ|. Thus, the conditions (19) and (20) hold if τ is bounded. For the implicit Euler method, we have for some positive number λ 0 . We observe from Figure 1 that the conditions (19) and (20) hold for τ > 0.
(a) (b) Finally, we need a technical result for the next two sections.

Lemma 2.
Let Assumptions 1 and 2 hold for the operator S := F (w + ). Then, there exists a positive number c R such that Proof. (i) Following the proof technique of Theorem 1 in [22] and using (17), we have and Using Assumption 2; the estimates in Equations (15), (16), (19), (20), (24); and the triangle inequality, we obtain with positive number Part (i) ensures an upper bound for the second term of the last formula. Hence, a similar upper bound for I + A k remains to be determined. To this end, we will use the inequality . Thus, by using (19) and (20), we obtain for some positive c, where the last inequality follows as in (24) and (25). Now, (26) combined with part (i) and the last inequality yield (22).

Convergence Rate for the Iterative RK-Type Regularization
To investigate the convergence rate of the RK-type regularization method (5) under the logarithmic source condition, the nonlinear operator F has to satisfy the local property in an open ball B ρ (w 0 ) of radius ρ around w 0 with w,w ∈ B ρ (w 0 ) ⊂ D(F). In addition, the regularization parameter k * is chosen according to the generalized discrepancy principle, i.e., the iteration is stopped after k * steps with where γ > 2−η 1−η is a positive number. Note that the triangle inequality yields In the sequel, we establish an error estimate that will be useful in deriving the logarithmic convergence rate. Theorem 1. Let Assumptions 1 and 2 be valid. Assume that problem (1) has a solution w + in B ρ 2 (w 0 ) and g ε fulfills (2). Furthermore, assume that the Fréchet derivative of F is scaled such that F (w) ≤ 1 for w ∈ B ρ 2 (w 0 ) and the source conditions (8) and (9) are fulfilled. Thereby, the iterative RK-type regularization is stopped according to the discrepancy principle (28). If v is sufficiently small, then there exists a constant c depending only on p and v such that for any 0 ≤ k < k * , and Proof. Using (5), we can show that Using the spectral theory and (14), we have Consequently, Equation (31) can be rewritten as where By recurrence and Equation (33), we obtain Moreover, it holds that We will prove by mathematical induction that e k ≤ c(ln(k + e)) −p and hold for all 0 ≤ k < k * with a positive constant c independent of k. Using the discrepancy principle (28), triangle inequality, and γ > 2 − η 1 − η , we can show that Using Proposition 1, Lemma 2, (34), and (39), it follows that withĉ 1 ≥ 1 2 c L + 2c R 1−η . By assumption S ≤ 1 (see Vainikko and Veterennikov [23], as cited in Hanke et al. [5]), we have and Therefore, Employing the assumption of the induction in Equations (37) and (38) into the third term of (45), we obtain (46) Similar to Equation (45) in [22], we have with a generic constant E < 2, which does not depend on k ≥ 1. Using (47), we can estimate (46) as The sum ∑ k−1 j=0 · is bounded because the integral is bounded with s := 1 2(k+1) from above by a positive constant E p independent of k. Thus, (45) becomes with c p =ĉ 1 E 2p E p . By assumption S ≤ 1 (see Vainikko and Veterennikov [23] as cited in Hanke et al. [5]), we have and Thus, Using Lemma 1, (40), (52), and (53), Equation (36) can be estimated as Se k ≤c 2 (k + 1) −1/2 (ln(k + e)) −p v + ε +ĉ 1 Using (47) and the assumption of the induction in Equations (37) and (38) into the third term of (54), we obtain The summation in (55) is bounded because, with s := 1 2(k+1) , the integral for some positive constant E p independently of k. Thus, (54) becomes and The discrepancy principles (28) and (29) provide Using (58), we obtain Applying (60) to (57), we obtain withĉ p = max{c p ,c p }.
Applying (60) to (58), we obtain We choose a sufficiently small v such that 1 + 1 ω c * v +ĉ p c 2 ≤ c. Thus, the induction is completed. Using (37), we can show that The second assertion is obtained by using (39) as follows: We are now in a position to show the logarithmic convergence rate for the iterative RK-type regularization under a logarithmic source condition, when the iteration is stopped according to the discrepancy principle (28).

Convergence Rate for the Modified Version of the Iterative RK-Type Regularization
The paper [13] contains a study of the modified iterative Runge-Kutta regularization method where ζ ∈ D(F) and Π = δ + τ k AF (w ε k ) * F (w ε k ). More precisely, it presents a detailed convergence analysis and derives Hölder-type convergence rates.
The aim in this section is to show convergence rates for (68) with the natural choice ζ = w 0 . We consider here the logarithmic source condition (9) with ϕ defined by (8), where v is small enough. That is, we deal with the following method: We work further under assumptions (27) and (28) with an appropriately chosen constant γ-compare inequality (2.10) in [13]. For the sake of completeness, we recall below the convergence result adapted to the choice ζ = w 0 (compare to Proposition 2.1 and Theorem 2.1 in [13]). Theorem 3. Let w + be a solution of (1) in B ρ 8 (w 0 ) with w 0 = w ε 0 and assume that g ε fulfills (2).
If the parameters α k , ∀k ∈ N 0 with ∞ ∑ k=0 α k < ∞ are small enough and if the termination index is defined by (28), then w ε k * → w † as ε → 0.
We state below a result on essential upper bounds for the errors in (69).
Theorem 4. Let Assumptions 1 and 2 hold for the operator S := F (w † ). Assume that problem (1) has a solution w + in B ρ 8 (w 0 ) and g ε fulfills (2). Assume that the Fréchet derivative of F is scaled such that F (w) ≤ 1 for w ∈ B ρ 8 (w 0 ) and that the parameters α k , ∀k ∈ N 0 with ∞ ∑ k=0 α k < ∞ are small enough. Furthermore, assume that the source condition (9) is fulfilled and that the modified RK-type regularization method (69) is stopped according to (28). If v is sufficiently small, then there exists a constant c depending only on p and v such that for any 0 ≤ k < k * , and Proof. First, we deduce an explicit formula for e k = w + − w ε k . We proceed similarly to the proof in Theorem 1, but this time, we need to take into account the additional term −α k (w ε k − w 0 ). Thus, the proof steps are as follows. I. We establish an explicit formula for the error e k : where the last equality follows from (32). We denoted q k = F(w ε k ) − F(w + ) − S(w ε k − w + ). Thus, (72) can be shortly rewritten as with Therefore, we obtain the following closed formula: From Proposition 1, (34), Lemma 2, (39), and (74), it follows that z k ≤ĉ e k Se k , for any 0 ≤ k ≤ k * , whereĉ is a positive constant. II. Following the technical steps of the proof of Theorem 1 in [22], one can similarly show by induction that there is a positive number c, such that the following inequalities hold for any 0 ≤ k ≤ k * : e k ≤ c(ln(k + e)) −p , Then, one can eventually obtain (70) and (71) as in the mentioned proof. Note that Theorem 1 in [22] is based on Proposition 1 in [22], which requires small enough parameters , for all n > k ≥ 1 (compare to (16) on page 4 in [22]). Since the smallest value of ln(k+e) ln(k+1+e) is about 0.84 (when k = 1), one can clearly find α k ∈ (0, 1) small enough so as to satisfy the imposed inequalities, e.g., a harmonic-type sequence such as α k = 1/(k + 2) r for some r > 1.
One can show convergence rates for the modified Runge-Kutta regularization method, as done in the previous section for the unmodified version.

Numerical Example
The purpose of the following numerical example is to verify the error estimates shown above. Define the nonlinear operator F : with the kernel function The noisy data is given by g ε (s) = exp(sin(πs)/π 2 ) + ε cos(100s), s ∈ [0, 1], and the exact solution is w * (t) = sin(πt). In order to demonstrate the results in Theorems 1 and 4, we consider Landweber, Levenberg-Marquardt (LM), Lobatto IIIC, and the Radau IIA methods, see Table 1 for the Butcher tableau.
The implementation in this section is the same as the one reported in [10,13]. The number of basis functions is 65 and the number of equidistant grid points is 150, while the parameter τ k is the harmonic sequence term (k + 1) 1.1 . As expected, the results in Figure 2 show that the curve of ln w + − w ε k lies below a straight line with slope −p, as suggested by (30) in Theorem 1 and (70) in Theorem 4.

Summary and Outlook
Up to now, the logarithmic convergence rate under logarithmic source condition has only been investigated for particular examples, namely, the Levenberg-Marquardt method (Böckmann et al. [21]) and the modified Landweber method (Pornsawad et al. [22]). Here, we extended the results to the whole family of Runge-Kutta-type methods with and without modification. For the future, it is still open to prove the optimal convergence rate under Hölder source condition for the whole family without modification.
Author Contributions: The authors P.P., E.R., and C.B. carried out jointly this research work and drafted the manuscript together. All the authors validated the article and read the final version. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.