Q-curve and area rules for choosing heuristic parameter in Tikhonov regularization

We consider choice of the regularization parameter in Tikhonov method if the noise level of the data is unknown. One of the best rules for the heuristic parameter choice is the quasi-optimality criterion where the parameter is chosen as the global minimizer of the quasi-optimality function. In some problems this rule fails. We prove that one of the local minimizers of the quasi-optimality function is always a good regularization parameter. For choice of the proper local minimizer we propose to construct the Q-curve which is the analogue of the L-curve, but on x-axis we use modified discrepancy instead of discrepancy and on the y-axis the quasi-optimality function instead of the norm of the approximate solution. In area rule we choose for the regularization parameter such local minimizer of the quasi-optimality function for which the area of polygon, connecting on Q-curve this minimum point with certain maximum points, is maximal. We also provide a posteriori error estimates of the approximate solution, which allows to check the reliability of parameter chosen heuristically. Numerical experiments on extensive set of test problems confirm that the proposed rules give much better results than previous heuristic rules. Results of proposed rules are comparable with results of the discrepancy principle and the monotone error rule, if last two rules use the exact noise level.


Introduction
Let A ∈ L (H, F) be a linear bounded operator between real Hilbert spaces H, F. We are interested in finding the minimum norm solution u * of the equation where noisy data f ∈ F are given instead of the exact data f * . The range R(A) may be nonclosed and the kernel N (A) may be non-trivial, so in general this problem is ill-posed. We consider solution of the problem Au = f by Tikhonov method (see [7,38]) where regularized solutions in cases of exact and inexact data have corresponding forms u + α = (αI + A * A) −1 A * f * , u α = (αI + A * A) −1 A * f and α > 0 is the regularization parameter. Using the well-known estimate u α − u + α ≤ [7,38]) and notations e(α) := u α − u * , arXiv:1809.02061v1 [math.NA] 6 Sep 2018 we have the error estimates e(α) ≤ e 1 (α) ≤ e 2 (α, f − f * ). ( We consider choice of the regularization parameter if the noise level for f − f * is unknown. The parameter choice rules which do not use the noise level information are called heuristic rules. Well known heuristic rules are the quasi-optimality criterion [3,12,23,24,25,27,37], L-curve rule [18,19], GCV-rule [9], Hanke-Raus rule [17], Reginska's rule [35]; about other rules see [21,28]. Heuristic rules are numerically compared in [4,12,21,28]. The heuristic rules give good results in many problems, but it is not possible to construct heuristic rule guaranteeing convergence u α − u * → 0 as the noise level goes to zero (see [2]). All heuristic rules may fail in some problems and without additional information about the solution, it is difficult to decide, is the obtained parameter reliable or not.
In the quasi-optimality criterion parameter α is chosen as the global minimizer of the function ψ Q (α) = α du α dα on certain interval [α N , α 0 ]. We propose to choose parameter from the set L min of local minimizers of this function from certain set Ω of parameters.
We will call the parameter α R in arbitrary rule R as pseudooptimal, if with relatively small constant c and we show that at least one parameter from set L min has this property. For the choice of proper parameter from the set L min some algorithms were proposed in [34], in the current work we propose other algorithms. We propose to construct Q-curve which is the analogue of the L-curve [18], but on x-axis we use modified discrepancy instead of discrepancy and on the y-axis the function ψ Q (α) instead of u α . For finding proper local minimizer of the function ψ Q (α) we propose the area rules on the Q-curve. The idea of proposed rules is that we form for every minimizer of the function ψ Q (α) certain function which approximates the error of the approximate solution and has one minimizer; we choose for the regularization parameter such local minimizer of ψ Q (α) for which the area of polygon, connecting this minimum point with certain maximum points, is maximal. The plan of this paper is as follows. In Section 2 we consider known rules for choice of the regularization parameter, both in case of known and unknown noise level. In Section 3 we prove that the set L min contains at least one pseudooptimal parameter. In Section 4 information about used test problems (mainly from [19,5], but also from [1,10,19,22,39]) and numerical experiments is given. In Section 5 we consider the Q-curve and area rule, in Section 6 further developments of the area rule. These algorithms are also illustrated by results of numerical experiments.
2. Rules for the choice of the regularization parameter 2.1. Parameter choice in the case of known noise level In case of known noise level δ , f − f * ≤ δ we use one of so-called δ -rules, where certain functional d(α) and constant b ≥ b 0 (b 0 depends on d(α)) is chosen and such regularization parameter α(δ ) is chosen which satisfies d(α) = bδ .

Parameter choice in the case of unknown noise level
A classical heuristic rule is the quasi-optimality criterion. In Tikhonov method it chooses α = α Q or α = α QD as the global minimizer of corresponding functions The Hanke-Raus rule finds parameter α = α HR as the global minimizer of the function In practice often L-curve is used. L-curve is log-log-plot of u α versus Au α − f . The points ( Au α − f , u α ) have often shape similar to the letter L and parameter α L which corresponds to the "corner point" is often a good parameter. In the literature several concrete rules for choice of the 'corner point' are proposed. In [35] parameter is chosen as the global minimizer of the function (below we use this rule with τ = 1). Another rule for choice of the corner point is the maximum curvature method ( [20,6]), where such parameter α is chosen for which the curvature of the L-curve as the function is maximal. Hereρ ,ξ ,ρ ,ξ are first and second order derivatives of functions log d D (α) and log u α . We propose also a new heuristic rule, where the global minimizer of the function is chosen for the parameter. We call this rule as the weighted quasioptimality criterion. In the following we will find the regularization parameter from the set of parameters where α 0 , q, α N are given. If in the discretized problem the minimal eigenvalue λ min of the matrix A T A is larger than α N , the heuristic rules above often choose parameter α N , which is generally not a good parameter. The works [24,25,27] propose to search the global minimum of the function ψ Q (α) in the interval [max (α N , λ min ), α 0 ]. We say that the discretized problem Au = f does not need regularization if e 1 (λ min ) ≤ 2 min α∈Ω, α≥λ min e 1 (α).
If the discretized problem does not need regularization then α = 0 or α = α N is the proper parameter while for α ≤ λ min we have Searching the parameter from the interval [max (α N , λ min ), α 0 ] means the a priori assumption that the discretized problem needs regularization. Note that if λ min > α N , then in general it is not possible to decide (without additional information about solution or about noise of the data), whether the discretized problem needs regularization or not.

Local minimum points of the function ψ Q (α)
In the following we investigate the function ψ Q (α) in (5) and show that at least one local minimizer of this function is the pseudooptimal parameter. We need some preliminary results.
In the following we define the local minimum points of the function ψ Q (α) on the set Ω (see (7)). We say that the parameter α k , 0 ≤ k ≤ N − 1 is the local minimum point of the Denote the local minimum points by m k , k = 1, . . . , K (K is the number of minimum points) and corresponding set by L min = {m k : m 1 > m 2 > ... > m K } .
The parameter α k , 0 < k < N is the local maximum point of the sequence ψ Q (α k ) if ψ Q (α k ) > ψ Q (α k+1 ) and there exists index j ≥ 1 so, that We denote by M k the local maximum point between the local minimum points m k+1 and Then by the construction Theorem 3. The following estimates hold for the local minimizers of the function ψ Q (α).
where c q := (ii) For arbitrary α 0 , α N we have Proof. For arbitrary parameters α ≥ 0, β ≥ 0 the inequalities and (8) lead to the estimate It is easy to see that while in case qα ≤ α ≤ α we have e 1 (α ) ≤ q −1 e 1 (α). Let α j * = α 0 q j * be the global minimizer of the function e 1 (α) on the set of the parameters Ω. Then α j * ∈ [M k , M k−1 ] for some k, 1 ≤ k ≤ K and this k defines index m with m k = α m . From (13) we get the estimate which together with (14) gives also estimate (12).
Now we prove the estimate (10). For the global minimum point α * of the function e 2 (α, f − f * ) the inequality α * ≥ α N holds, while for α < α N we have In the case α * ≤ α 0 we get similarly as in the proof of estimate (12) that min α∈L min (10) holds. Consider the case α * > α 0 . Then and for each local minimum point m k , α N ≤ m k ≤ α 0 the inequalities Therefore the inequality (10) holds also in this case. For is well-known (see [7,38]) and the estimate (11) follows immediately from (10). Remark 5. The inequality (12) holds also in the case if the noise of the f is not finite but Remark 6. Use of the inequality (9) enables to prove the analogue of Theorem 3 for set L min of local minimizers of the function ψ QD (α): then the inequality (12) holds, where . In choice of the regularization parameter we may exclude from the observation some local minimizers. It is natural to assume that α N is so small that with small ε > 0. Then the following theorem holds.
Proof. Let α * i , i = 1, 2 be global minimizers of the functions e 1 (α) and e 2 (α, f − f * ) respectively. We consider separately 3 cases. If m k 0 ≤ α * 1 we get similarly to the proof of Theorem 3 the estimate min α∈L min ,α≥m k 0 and now we can prove analogically to the proof of the weak quasioptimality of the modified discrepancy principle ( [31]) that under assumption α * 2 ≤ m k 0 the error estimate holds. Now the assertion 1 of Theorem 7 follows from the inequalities (16)- (18).

On test problems and numerical experiments
We made numerical experiments for local minimizers of the function ψ Q (α) using three sets of test problems. The first set contains 10 well-known test problems from Regularization Toolbox [19] and the following 6 Fredholm integral equations of the first kind (discretized by the midpoint quadrature formula ) • ursell [19]: The second set of test problems are well-known problems from [5] : gauss, hilbert, lotkin, moler, pascal, prolate. As in [5], we combined these six n × n matrices with 6 solution vectors For getting the third set of test problems we combined the matrices of the first set of test problems with 6 solutions of the second set of test problems. Numerical experiments showed that performance of different rules depends essentially on eigenvalues of the matrix A T A. We characterize these eigenvalues via three indicators: the value of minimal eigenvalue λ min , by the value N 1 , showing number of eigenvalues less than α N and by the value Λ, characterizing the density of location of eigenvalues on the interval We characterize the smoothness of the solution by value where f − f * = 10 −6 . Table 1 contains the results of characteristics of the matrix A T A in case n = 100, α N = 10 −18 .
In all tests discretization parameters n ∈ {60, 80, 100, 120, 140, 160, 180} were used. We present the results of numerical experiments in tables for n = 100. Since the performance of rules generally depends on the smoothness p of the exact solution in (1), we complemented the standard solutions u * of (now discrete) test problems with smoothened solutions |A| p u * , p = 2 computing the right-hand side as A(|A| p u * ). Results for p = 2 are given in Table 7, in all other tables and figures p = 0. After discretization all problems were scaled (normalized) in such a way that the norms of the operator and the right-hand side were 1. All norms here and in the text below are Euclidean norms. On the base of exact data f * we formed the noisy data f , where f − f * has values 10 −1 , 10 −2 , ..., 10 −6 , noise f − f * has normal distribution and the components of the noise were uncorrelated. We generated 20 noise vectors and used these vectors in all problems. We search the regularization parameter from the set Ω, where α 0 = 1, q = 0.95 and N is chosen so that α N ≥ 10 −18 > α N+1 . To guarantee that calculation errors do not influence essentially the numerical results, calculations were performed on geometrical sequence of decreasing α-s and finished for largest α with d MD (qα) > d MD (α), while theoretically the function d MD (α) is monotonically increasing. Actually this precautionary measure was needed only in problem groetsch2, calculations on α > α N were finished only in this problem. Since in model equations the exact solution is known, it is possible to find the regularization parameter α * , which gives the smallest error on the set Ω . For every rule R the error ratio describes the performance of the rule R on this particular problem. To compare the rules or to present their properties, the following tables show averages and maximums of these error ratios over various parameters of the data set (problems, noise levels δ ). We say that the heuristic rule fails if the error ratio E > 100. In addition to the error ratio E we present in some cases also error ratios The results of numerical experiments for local minimizers α ∈ L min of the function ψ Q (α) are given in the Table 2. For comparison the results of δ -rules with δ = f − f * are presented in the columns 2-4. Columns 5 and 6 contain respectively the averages and maximums of error ratios E for the best local minimizer α ∈ L min . The results show that for many problems the Tikhonov approximation with the best local minimizer α ∈ L min is even more accurate than with the δ -rules parameters α ME , α MEe or α DP . Tables 1, 2 show also that for rules ME and MEe the average error ratio E may be relatively large for problems where Λ is large and most of eigenvalues are smaller than α N , while in this case min α∈Ω e(α) may be essentially smaller than min α∈Ω e 2 (α, f * − f ). In these problems the discrepancy principle gives better parameter than ME and MEe rules. Columns 7 and 8 contain the averages and maximums of cardinalities |L min | of sets L min (number of elements of these sets). Note that number of local minimizers depends on parameter q (for smaller q the number of local minimizers is smaller) and on length of minimization interval determined by the parameters α N , α 0 . The number of local minimizers is smaller also for larger noise level. Columns 9 and 10 contain the averages and maximums of values of constant C in the a posteriori error estimate (12). The value of C and error estimate (12) allow to assert, that in our test problems the choice of α as the best local minimizer in L min guarantees that error of the Tikhonov approximation has the same order as min α N ≤α≤α 0 e 1 (α). Note that over all test problems the maximum of error ratio E1 for the best local minimizer in L min and for the discrepancy principle were 1.93 and 9.90 respectively. This confirm the result of Theorem 3 that at least one minimizer of the function ψ Q (α) is a good regularization parameter.

Q-curve and triangle area rule for choosing heuristic regularization parameter
We showed in previous section that at least one local minimizer of the function ψ Q (α) is pseudooptimal parameter and we may omit small local minimizers α, for which d MD (α) is only slightly larger than d MD (α N ). We propose to construct for parameter choice the Qcurve The Q-curve figure uses log-log scale with functions d MD (α) and ψ Q (α) on the xaxis and y-axis respectively. The Q-curve can be considered as the analogue of L-curve, where functions Au α − f and u α = −α −1 A * (Au α − f ) are replaced by functions B α (Au α − f ) and −α −1 A * B 2 α (Au α − f ) (see (4)) respectively. We denoted MD (α) := log 10 d MD (α), ψ Q (α) := log 10 ψ Q (α). For many problems the curve (d MD (α),ψ Q (α)) (or a part of this) has the form of letter L or V and we choose the minimizer at the "corner" point of L or V. We use the common logarithm instead of natural logarithm, while then the Q-curve allows easier to estimate the supposed value of the noise level. On the figures 1-8 n = 100 is used, on the figures 9, 10 n = 60. On the figures 1-4 the L-curves and Q-curves are compared for two problems, the global minimizer α opt of the function e 1 (α) is also presented. Note that in problem baart λ min < α N and in problem deriv2 λ min > α N .
In most cases one can see on the Q-curve only one clear "corner" with one local minimizer. If the "corner" contains several local minimizers, we recommend to choose such local minimizer, for which the sum of coordinates of corresponding point on the Qcurve is minimal. If Q-curve has several "corners" we recommend to use the very right of them. Actually, it is useful to present in parameter choice besides figures for every local minimizer m k of the function ψ Q (α) also coordinates of point (d MD (m k ),ψ Q (m k )) and sums of coordinates.
For finding proper local minimizer of the function ψ Q (α) we present now a rule which works well for all test problems from set 1. The idea of rule is to search proper local    minimizer m k constructing certain triangles on the Q-curve and finding which of them has the maximal area. For parameter α corresponds a point P(α) on the Q-curve with corresponding coordinates d MD (α),ψ Q (α) . For every local minimizer m k of the function ψ Q (α) corresponds a triangle T (k, r(k), l(k)) with vertices P(m k ), P(M r(k) ) and P(M l(k) ) on the Q-curve, where indices r(k) and l(k) correspond to the largest local maximums of the function ψ Q (α) on two sides of the local minimum m k : Triangle area rule (TA-rule). We choose for the regularization parameter such local minimizer m k of the function ψ Q (α) for which the area of the triangle T (k, r(k), l(k)) is the largest.
We present on the Figures 5-7 examples of Q-curves and triangle T (k, r(k), l(k)) with largest area, the TA-rule chooses for the regularization parameter corresponding minimizer. In some problems the function ψ Q (α) may be monotonically increasing as for problem groetsch2 (Figure 8), then the function ψ Q (α) has only one local minimizer α N . Then vertices P(M l(k) ) and P(m k ) coincide and area of corresponding triangle is zero. Then this is the only triangle, the TA-rule chooses for the regularization parameter α N . The results of the numerical experiments for test set 1 ( n = 100 ) for the TA-rule and some other rules (see Section 2.2)    are given in Tables 3 and 4. These results show that the TA-rule works well in all these test problems, the accuracy is comparable with δ -rules (see Table 2), but previous heuristic rules fail in some problems. Note that average of the error ratio increases for decreasing noise level.
For example, for f − f * ∈ {10 −1 , 10 −2 , 10 −5 , 10 −6 } corresponding error ratios E were 1.47, 1.49, 1.78 and 2.08 respectively. Let us comment other heuristic rules. The accuracy of the quasi-optimality criterion is for many problems the same as for the TA-rule, but this rule fails in problem heat. Characteristic feature of the problem heat is that location of the eigenvalues in the interval [α N , 1] is sparse and only some eigenvalues are smaller than α N (see Table 1). The weighted quasioptimality criterion behaves in a similar way as the quasioptimality criterion, but is more accurate in problems where λ min ≤ α N ; if λ min > α N , the quasioptimality criterion is more accurate. The rule of Hanke-Raus may fail in test problems with large Λ and for other problems the error of the approximate solution is in most problems approximately two times larger than for parameter chosen by the quasi-optimality principle. The problem in this rule is that it chooses too large parameter compared with the optimal parameter. However, HR-rule is stable in the sense that the largest error ratio E2 is relatively small in all considered test problems. Reginska's rule may fail in many problems but it has the advantage that it works   Table 4 shows also that in test problems set 1 from considered rules the TA-rule is the most accurate rule.
Note that figure of the Q-curve enables to estimate the reliability of chosen parameter. If the Q-curve has only one "corner", then chosen parameter is quasioptimal with small constant C, if λ min < α N , but in case λ min ≥ α N it is quasioptimal under assumption that the problem needs regularization.

Further developments of the area rule
The TA-rule may fail for problems which do not need regularization, if the function ψ Q (α) is not monotonically increasing. In this case the TA-rule chooses parameter α ≥ λ min , but parameter α < λ min would be better. For example, the TA-rule fails for matrix Moler in some cases. Let us consider now the question, in which cases regularization parameter α N is good. If the function ψ Q (α) is monotonically increasing then the function ψ Q (α) has only one local minimizer m 1 = M 1 = α N and then for parameter α N we have the error estimate Theorem 3) can be computed a posteriori and this value is the smaller the faster the function ψ Q (α) increases. We can take α N for the regularization parameter also in the case if the condition holds while one can show similarly to the proof of Theorem 3 that T (α N ) ≤ c 0 c q ln( α 0 α n ) and the error of the regularized solution is small. For problems which do not need regularization we can improve the performance of the TA-rule searching proper local minimizer smaller or equal than α HQ := max{α HR , α Q }, where α HR , α Q are global minimizers of functions ψ HR (α) and ψ Q (α) respectively on the interval [max (α N , λ min ), α 0 ].
These ideas enable to formulate the following upgraded version of the TA-rule. Triangle area rule 2 (TA-2-rule). We fix a constant c 0 , 1 ≤ c 0 ≤ 2. If condition (19) holds, we choose parameter α N . Otherwise choose for the regularization parameter such local minimizer m k ≤ α HQ of the function ψ Q (α) for which the area of triangle T (k, r(k), l(k)) is largest.
Results of numerical experiments for the rule TA-2 with the discretization parameter n = 100 and problem sets 1-3 are given in Tables 5 and 6 (columns 2 and 3). The results show that rule TA-2 works well in all considered testsets 1-3. However, rule TA-2 may fail in some other problems which do not need regularization. Such example is problem with matrix moler and solution x i = sin (12π(i − 1)/n), where the rule TA-2 fails if the noise level is below 10 −4 ; but in this case all other considered heuristic rules fail too.
Rules TA and TA-2 fail in problem heat in some cases for discretization parameter n = 60. Figures 9, 10 show the form of Q-curve in problem heat with n = 60. Function ψ Q (α)  has two local minimizers with corresponding points P(m 1 ) and P(m 2 ) on the Q-curve and 3 local maximum points P(M k ), k = 0, 1, 2. On Figure 9 Rule TA-2 chooses local minimizer corresponding to the point P(m 2 ), but then the error ratios are large: E = 20.3, E2 = 18.34.
Let g[α 1 , α 2 , ..., α k ](α), k > 2 be parametric representation of broken line connecting points P(α 1 ), P(α 2 ), . . . , P(α k ), thus In triangle rule certain points P(M l(k) ), P(m k ), P(M r(k) ) are connected by the broken line t 1 (α) = g[M l(k) , m k , M r(k) ](α) which approximates the error functionẽ 1 (α) = log 10 (e 1 (α)) well, if m k is the "right" local minimizer. By construction of function t 1 (α) we use only 3 points on the Q-curve. We will get a more stable rule if the form of the Q-curve has more influence to the construction of approximates to the error functionẽ 1 (α). Let {i(1), i(2), ..., i(n1)} and { j(1), j(2), ..., j(n2)} be the largest sets of indices, satisfying the inequalities It is easy to see that i(n1) = l(k) and j(n2) = r(k). For approximating the error functioñ e 1 (α) we propose to connect points P(M i(n1) ), ..., P(M i(1) ), P(m k ), P(M j(1) ), ..., P(M j(n2) ) by broken line t 2 (α) = g[M i(n1) , ..., M i(1) , m k , M j(1) , ..., M j(n2) ](α) and to find for every m k the area S 2 (k) of polygon surrounded by lines T 2 (α) = max{t 2 (α), g[M i(n1) , M j(n2) ](α)} and t 2 (α). The second possibility is to approximate the error functionẽ 1 (α) by the curve t 3 (α) = max{t 2 (α),ψ Q (α)} and to find S 3 (k) as the area of polygon surrounded by broken lines t 3 (α) and curve T 3 (α) = max{T 2 (α),ψ Q (α)}. Note that functions t i (α), i = 1, 2, 3 are monotonically increasing if α > m k , and monotonically decreasing if α < m k . Area rules 2 and 3. We fix constant c 0 , 1 ≤ c 0 ≤ 2. First we choose local minimizer m k ≤ α HQ , for which the area S i (k), i ∈ {2, 3} is largest. We take for the regularization parameter the smallest m k 0 ≤ m k , satisfying the condition (compare with (19)) Let us consider Figure 9. Area rules 2 and 3 work in problem heat well for every n and all α N = 10 −k , 12 ≤ k ≤ 24, but in some other problems accuracy of area rules 2 and 3 (see columns 4-7 of Tables 5, 6) is slightly worser than for rule TA-2. The advantage of area rule 3, as compared to area rule 2, is to be highlighted in problem heat if all noise of the right hand side is placed on one eigenelement (then we use the condition m k ≤ α HQ only in case λ min > α N ). Then area rule 3 did not fail if n ≥ 80 and α N ≤ 10 −20 . So we can say, the more precisely we take into account the form of the Q-curve in construction of the approximating function for the error functioñ e 1 (α), the more stable is the rule.
Based on the above rules it is possible to formulate a combined rule, which chooses the parameter according to the rule TA-2 or area rule 3 in dependence of certain condition.
Area rule 4 (Combined area rule). Fix constant c 0 , 1 ≤ c 0 ≤ 2, b ≥ 0. Let local minimizer m k be chosen by the rule TA-2. If max m k ≤α≤M r(k)ψ we take m k for the regularization parameter, otherwise we choose regularization parameter by rule 3. Note that combined rule coincides with rule TA-2, if b = 0 and with area rule 3, if b = ∞. Experiments of combined rule with c 0 = 2, b = 1 (columns 8 and 9 in Tables 5, 6)) show that accuracy of this rule is almost the same as in triangle rule, but unlike the TA-2 rule, it works well also in the problem heat for all n and α N . Although, in some cases, in test set 3 the error ratio E > 100 for rule 4, the high qualification of the rule is characterized by fact, that over all problems sets 1-3 the largest error ratio E1 was 16.91 (5.06 for set 1) and the largest error ratio E2 was 4.67 (2.62 for set 1). Numerical experiments show that it is reasonable to use parameter b ∈ (0.8, 1.2]. We studied the behavior of area rules for different α N = 10 −k , 12 ≤ k ≤ 24. The results were similar to results of Tables 5 and 6, but for smaller α N the error ratios were 2-3% smaller than for α N = 10 −18 and for larger α N the error ratios were about 5% larger than in Tables 5, 6.  The Table 7 gives results of the numerical experiments in the case of smooth solution, p = 2. We see that combined rule worked well also in this case, no failure. Remark 8. It is possible to modify the Q-curve. We may use the function ψ QD (α) instead of function ψ Q (α) and find proper local minimizer of the function ψ QD (α). Unlike the quasioptimality criterion the use of function ψ QD (α) in the Q-curve and in the area rule does not increase the amount of calculations, while approximation u 2,α is needed also in computation of d MD (α). We can use in these rules the function d ME (α) instead of d MD (α), it increases the accuracy in some problems, but the average accuracy of the rules is almost the same. In case of nonsmooth solutions we can modify the Q-curve method and area rule, using the function d D (α) instead of d MD (α). In this case, we get even better results for p = 0 but for p = 2, the error ratio E is on average 2 times higher.
Note that if solution is smooth, then L-curve rule and Reginska's rule often fail, but replacing in these rules the functions Au α − f and u α = −α −1 A * (Au α − f ) by functions B α (Au α − f ) and −α −1 A * B 2 α (Au α − f ) (then Reginska's rule modifies to minimization of the function ψ WQ (α) (see (6)) respectively gives often better results.
In the case of a heuristic parameter choice, it is also possible to use the a posteriori estimates of the approximate solution, which, in many tasks, allows to confirm the reliability of the parameter choice. Let α H be the regularization parameter from some heuristic rule and α * be the local minimizer of the function e 1 (α) on the set Ω. Then in case α * ≥ α H the error estimate u α H − u * ≤ (1 + T (α H , α * )) e 1 (α * ) ≤ q −1 (1 + T 1 (α H )) min α e 1 (α) (20) holds where T 1 (α H ) = max α≥α H ,α∈Ω T (α H , α). Using the last estimate, we can prove similarly to the Theorem 7 that if α N is so small that d MD (α N ) ≤ (1 + ε) f − f * , then . If values T 1 (α H ) and b what we find a posteriori, are small (for example b ≤ 2 and T 1 (α H ) ≤ 9), then this estimate allows to argue that error of approximate solution for this parameter is not much larger than the minimal error. The conditions b ≤ 2 , T 1 (α H ) ≤ 9 were satisfied in set 1 of test problems in combined rule for 73% of cases and inequalities b ≤ 2 , T 1 (α H ) ≤ 4 for 61% of cases. The reason of failure of heuristic rule is typically that chosen parameter is too small. To check this, we can use the error estimate (20). If T 1 (α H ) is relatively small (for example T 1 (α H ) ≤ 9), then estimate (20) allows to argue that the regularization parameter is not chosen too small. In set 1 of test problems the conditions T 1 (α H ) ≤ 9 and T 1 (α H ) ≤ 4 were satisfied in 97% and in 82% of cases respectively. We finish the paper with the following conclusion. For the heuristic choice of the regularization parameter we recommend to choose the parameter from the set of local minimizers of the function ψ Q (α) or the function ψ QD (α). For choice of the parameter from the local minimizers we proposed the Q-curve method and different area rules. The proposed rules gave much better results than previous heuristic rules on extensive set of test problems. Area rules fail in very few cases in comparison with previous rules, and the accuracy of these rules is comparable even with the δ -rules if the exact noise level is known. In addition, we also provided a posteriori error estimates of the approximate solution, which allows to check the reliability of parameter chosen heuristically.