Fuzzy Adaptive Parameter in the Dai–Liao Optimization Method Based on Neutrosophy

Abstract

The impact of neutrosophy has increased rapidly in many areas of science and technology in recent years. Furthermore, numerous applications of the neutrosophic theory have become more usual. We aim to use neutrosophy to enhance Dai–Liao conjugate gradient (CG) iterative method. In particular, we suggest and explore a new neutrosophic logic system intended to compute the essential parameter t required in Dai–Liao CG iterations. Theoretical examination and numerical experiments signify the effectiveness of the introduced method for controlling t. By incorporation of the neutrosophy in the Dai–Liao conjugate gradient principle, we established novel Dai–Liao CG iterations for solving large-scale unconstrained optimization problems. Global convergence is proved under standard assumptions and with the use of the inexact line search. Finally, computational evidence shows the computational effectiveness of the proposed fuzzy neutrosophic Dai–Liao CG method.

1. Introduction and Background Results

Numerous iterative methods have been developed for solving the large-scale unconstrained optimization problem

$$\mathrm{minimize}f\left(\mathbf{x}\right),\mathbf{x}\in {\mathbb{R}}^n,$$

(1)

in which $f:{\mathbb{R}}^n\to \mathbb{R}$ is continuously differentiable function bounded below. Continuing well-established notation, g${}_k=\mathbf{g}\left({\mathbf{x}}_k\right)=\nabla f\left({\mathbf{x}}_k\right)$ stands for the gradient vector of f at the actual iterative point ${\mathbf{x}}_k$, and further ${\mathbf{y}}_{k-1}={\mathbf{g}}_k-{\mathbf{g}}_{k-1}$ and ${\mathbf{s}}_{k-1}={\mathbf{x}}_k-x_{k-1}$. Utilizing the extended conjugacy condition

$${\mathbf{d}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}=-t{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1},t>0,$$

(2)

Dai and Liao in [1] suggested the conjugate gradient (CG) iterations

$$x_{k+1}={\mathbf{x}}_k+{\alpha }_k{\mathbf{d}}_k,$$

(3)

in which ${\mathbf{x}}_k$ is the last calculated iteration, ${\mathbf{x}}_{k+1}$ is the new iterative point, ${\alpha }_k$ is a positive step size parameter defined as the output of a proper inexact line search, and ${\mathbf{d}}_k$ is a descent direction. The search directions $\{{\mathbf{d}}_k,k\ge 0\}$ are generated by the recurrent rule

$${\mathbf{d}}_k=\left\{\begin{array}{cc}-{\mathbf{g}}_0,\hfill & k=0,\hfill \\ -{\mathbf{g}}_k+{\beta }_k^{\mathrm{DL}}{\mathbf{d}}_{k-1},\hfill & k\ge 1,\hfill \end{array}\right.$$

(4)

where ${\beta }_k^{\mathrm{DL}}$ is the CG coefficient that describes the type of CG method according to the general rule

$${\beta }_k^{\mathrm{DL}}=\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}-t\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}},$$

(5)

wherein $t>0$ is an appropriate scalar. The Dai–Liao (DL) method guarantees global convergence for uniformly convex objective functions. These results have attracted considerable attention, leading to the creation of several methods based on various patterns for defining ${\beta }_k$. Most of these methods are developed by modifying the conjugate gradient parameter ${\beta }_k^{\mathrm{DL}}$ [2,3,4,5,6,7,8,9,10,11]. For more details, see the survey on the DL family of nonlinear CG methods in [12]. One of rules for defining ${\beta }_k$ is denoted as ${\beta }_k^{\mathrm{MHSDL}}$ and defined in [7] by

$${\beta }_k^{\mathrm{MHSDL}}=\frac{{\mathbf{g}}_k^{\mathrm{T}}\widehat{{\mathbf{y}}_{k-1}}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}-t\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}},$$

(6)

such that $\widehat{{\mathbf{y}}_{k-1}}={\mathbf{g}}_k-\frac{\parallel {\mathbf{g}}_k\parallel }{\parallel {\mathbf{g}}_{k-1}\parallel }{\mathbf{g}}_{k-1}$ and $t>0$ is as in (5).

Due to the large influence of the quantity t on the numerical results generated by the DL class of CG methods [13], one of the most common issues is the determination of an appropriate value t. We can distinguish two research directions based on the previous results in determining proper values of t in the DL-established CG iterations. The first line of research consists of a group of DL methods that aim to find a suitable constant value for t during iterations [1,2,6,7,8], while the second line consists of a group of DL methods that propose a suitable control in recalculating t in each iteration. In this research, we prioritize the second research stream: find values of t that change appropriately across iterations. The quantity t determined in the kth iterative step will be denoted by $t_k^{\left(i\right)}$, where i is a variant of the algorithm for defining t.

Some of the most important adaptive choices for the DL parameter $t_k$ will be presented in the rest of this section. Hager and Zhang in [14,15] proposed the CG-DESCENT method, which is classified into the group of the DL CG methods (5) defined by $t\equiv t_k^{\left(1\right)}$ and

$$t_k^{\left(1\right)}=2\frac{\parallel {\mathbf{y}}_{k-1}{\parallel }^2}{{\mathbf{y}}_{k-1}^{\mathrm{T}}{\mathbf{s}}_{k-1}}.$$

(7)

Dai and Kou in [16] proposed the DK method, which is based on the CG coefficient ${\beta }_k^{\mathrm{DK}}$ of the form

$${\beta }_k^{\mathrm{DK}}=\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{y}}_{k-1}^{\mathrm{T}}{\mathbf{d}}_{k-1}}-\left({\tau }_k+\frac{\parallel {\mathbf{y}}_{k-1}{\parallel }^2}{{\mathbf{y}}_{k-1}^{\mathrm{T}}{\mathbf{s}}_{k-1}}-\frac{{\mathbf{y}}_{k-1}^{\mathrm{T}}{\mathbf{s}}_{k-1}}{\parallel {\mathbf{s}}_{k-1}{\parallel }^2}\right)\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}.$$

(8)

In the equality (8), the parameter ${\tau }_k$ is defined utilizing the self-scaling memoryless BFGS method. It is also obvious from (8) that the DK method is involved in the DL CG class of methods based on the particular value $t\equiv t_k^{\left(2\right)}$, which is defined by

$$t_k^{\left(2\right)}={\tau }_k+\frac{\parallel {\mathbf{y}}_{k-1}{\parallel }^2}{{\mathbf{y}}_{k-1}^{\mathrm{T}}{\mathbf{s}}_{k-1}}-\frac{{\mathbf{y}}_{k-1}^{\mathrm{T}}{\mathbf{s}}_{k-1}}{\parallel {\mathbf{s}}_{k-1}{\parallel }^2}.$$

(9)

Babaie-Kafaki and Ghanbari in [17] gave the subsequent two rules for computing t in (5):

$$t_k^{\left(3\right)}=\frac{{\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\parallel {\mathbf{s}}_{k-1}{\parallel }^2}+\frac{\parallel {\mathbf{y}}_{k-1}\parallel }{\parallel {\mathbf{s}}_{k-1}\parallel }$$

(10)

and

$$t_k^{\left(4\right)}=\frac{\parallel {\mathbf{y}}_{k-1}\parallel }{\parallel {\mathbf{s}}_{k-1}\parallel }.$$

(11)

Andrei in [18] originated a new DL class, denoted DLE, where $t\equiv t_k^{\left(5\right)}$ is defined by

$$t_k^{\left(5\right)}=\frac{{\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\parallel {\mathbf{s}}_{k-1}{\parallel }^2}.$$

(12)

A special place in the DL iterations is occupied by the DL method, which is defined by

$$t_k^{\left(6\right)}=\upsilon \frac{\parallel {\mathbf{y}}_{k-1}{\parallel }^2}{{\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}},$$

(13)

where $\upsilon >\frac{1}{4}$ is a constant that is defined according to the sufficient descent condition

$${\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_k\le -c{\parallel {\mathbf{g}}_k\parallel }^2,\forall k\ge 0,$$

(14)

such that $c>0$ is a quantity independent of the cost function convexity and the line search rule (for more details, see [19]).

Lotfi and Hosseini in [20] suggested the subsequent rule

$$t_k^{\left(7\right)}=max\left\{t_k^{\left(7^{*}\right)},t_k^{\left(6\right)}\right\},$$

(15)

where

$$t_k^{\left(7^{*}\right)}=\frac{\left(1-{\hslash }_k\parallel g_{{k-1\parallel }^r}\right){\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{g}}_k+\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{y}}_{k-1}^{\mathrm{T}}{\mathbf{s}}_{k-1}}{\hslash }_k\parallel {\mathbf{g}}_{k-1}{\parallel }^r{\parallel {\mathbf{s}}_{k-1}\parallel }^2}{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}+\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\hslash }_k\parallel {\mathbf{g}}_{k-1}{\parallel }^r{\parallel {\mathbf{s}}_{k-1}\parallel }^2},$$

(16)

$${\hslash }_k=C+max\left\{-\frac{{\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\parallel {\mathbf{s}}_{k-1}{\parallel }^2},0\right\}{\parallel {\mathbf{g}}_{k-1}\parallel }^{-r},$$

(17)

and $C,r$ are positive constants.

Ivanov et al. in [21] proposed a variant of the Dai–Liao CG method (6), known as the Effective Dai–Liao (EDL) CG method, where $t\equiv t_k^{\left(8\right)}$ is determined as

$$t_k^{\left(8\right)}=\frac{{∥{\mathbf{g}}_k∥}^2}{max\left\{1,{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{g}}_k\right\}+\left(max\left\{0,\frac{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{g}}_k}{\parallel {\mathbf{g}}_k{\parallel }^2}\right\}+1\right){∥{\mathbf{g}}_k∥}^2}.$$

(18)

Experiments conducted in [21] confirm that EDL iterations outperform many existing CG variants.

Our motivation is to explore principles of neutrosophy to determine the crucial parameter t in the Dai–Liao CG method. Numerical experience clearly shows improvements in the behavior of the DL CG method based on the fuzzy parameter t established in this way. The justification for using the fuzzy values $t_k$ defined on the basis of neutrosophic logic lies in the fact that there is neither a best value for that parameter nor a clear rule for its determination. Our underlying motivation is based on the fact that the neutrosophic logic system is an effective tool to use the behavior of the goal function through iterations and thus avoid divergence or slow convergence. Effective tracking of changes in the objective function value is enabled by properly defined if–then rules and suitable membership functions.

The sections of the paper are arranged as follows. Introduction, basic details of motivation, and a brief review of obtained results are presented in Section 1. Section 2 briefly describes the most important related results about neutrosophy, DL methods, and motivation. A neutrosophic-based control for defining appropriate changeable values $t_k$ is proposed in Section 3. Moreover, we present details of the FDL method in the same section. The global convergence behavior of the FDL method is examined in Section 4. A numerical comparison of the FDL method with the main standard DL methods is presented in Section 5, and a comparison with some known variations of the DL class of methods is also provided. Final conclusions are presented in the concluding section.

2. Related Results about Neutrosophy and Motivation

A fuzzy set theory utilizes a membership function (MF) ${\mathrm{T}}_{\Psi }(\lambda )\in [0,1],\lambda \in \Lambda$ in the universe $\Lambda$, which defines the degree of membership of $\lambda$ in $\Psi \subset \Lambda$ [22]. An intuitionistic fuzzy set (IFS) $\Psi$ is based on membership and non-membership functions $T_{\Psi }(\lambda ),F_{\Psi }(\lambda )\in [0,1],\lambda \in \Lambda$ [23], which satisfy ${\mathrm{T}}_{\Psi }(\lambda ),{\mathrm{F}}_{\Psi }(\lambda ):\Lambda \to [0,1]$ and are jointly correlated by $0\le {\mathrm{T}}_{\Psi }(\lambda )+{\mathrm{F}}_{\Psi }(\lambda )\le 1$. The IFS theory was extended in [24,25] by the neutrosophic theory. The key innovation in Neutrosophy is based on the indeterminacy $\mathrm{I}(\lambda )$. As a consequence of such an approach, a phenomenon in a neutrosophic set is determined by three individualistic MFs [24,25]: the truth-MF $\mathrm{T}(\lambda )$, the indeterminacy-MF $\mathrm{I}(\lambda )$, and the falsity-MF $\mathrm{F}(\lambda )$. Because of independence between the three MFs, the neutrosophic logic is established on the symmetry involved in the ordered triple $(\mathrm{T},\mathrm{I},\mathrm{F})$ and the inequality $0\le \mathrm{T}+\mathrm{I}+\mathrm{F}\le 3$. Clearly, $\mathrm{T}$ is the symmetric pole to its opposite pole $\mathrm{F}$ relative to $\mathrm{I}$, which represents the center of symmetry between $\mathrm{T}$ and $\mathrm{F}$ [26]. The same observation is valid for refined neutrosophic sets that assume two refined indeterminacies ${\mathrm{I}}_1$ and ${\mathrm{I}}_2$ between extremes $\mathrm{T}$ and $\mathrm{F}$ [27]. The MFs of a neutrosophic set $\Psi$ satisfy ${\mathrm{T}}_{\Psi }(\lambda ),{\mathrm{I}}_{\Psi }(\lambda ),{\mathrm{F}}_{\Psi }(\lambda ):\Lambda \to [0,1]$, which is based on their independence and implies $0\le {\mathrm{T}}_{\Psi }(\lambda )+{\mathrm{I}}_{\Psi }(\lambda )+{\mathrm{F}}_{\Psi }(\lambda )\le 3$ and enables a symmetry between them. In [28], the authors originated a neutrosophic-based multiple criteria decision making procedure which is established on the introduced symmetry estimate. A complete overview of neutrosophic theory and applications is available in the exhaustive collections of papers [29,30]. A useful application of neutrosophic logic in natural language processing was discussed in [27].

The advantages of the NL approach over the FL and IFL in improving the gradient descent method are discussed in [31].

It is obvious that there is no single solution for determining the parameter $t_k$ in DL iterations (5). All previous solutions are based on specific rules. Moreover, it is clear that there are infinite possibilities for new rules in defining $t_k$. Our intention is to improve the behavior of the DL class for solving unconstrained nonlinear optimization problems with the support of an appropriate neutrosophic logic system. The indeterminacy and uncertainty in predicting an optimal value of $t_k$ justify the utilization of fuzzy quantities as well as neutrosophic principles in its determination. A specific convenience in the application of neutrosophy to the determination of the DL parameter arises from the restriction $0\le t_k\le 1$. Moreover, the neutrosophic logic system is a suitable tool for using the behavior of the objective function in determining required parameters with the help of appropriately defined if–then rules and suitable membership functions.

Application of fuzzy logic controllers in defining some parameters in the development of some recurrent neural network dynamical systems has been a popular topic in recent years [32,33,34,35,36].

We expect that the symmetry principles involved in the basis of neutrosophic logic (see [27,31] for more details) and tracking the prior values of the objective function will lead to better numerical results. Such expectations were confirmed by the results obtained through a series of numerical experiments.

Neutrosophic logic was applied in [31] in regulating proper step sizes for a class of accelerated gradient descent optimization methods. The approach in [31] assumes an additional fuzzy parameter that stabilizes the behavior of an important class of gradient descent family. Motivated by that approach, in this research, we apply neutrosophy to enhance the performances of DL methods. Based on the review and analysis of the class of DL methods, we propose a new method for determining $t_k$. The proposed method defines $t_k$ as the output generated by an appropriate neutrosophic logic controller (NLC). Our idea is to replace the classical parameter $t_k$ in (5) by an adaptive neutrosophic logic parameter ${\nu }_k$, determined as the output of the NLC. Since $0\le t_k\le 1$, we decided to define $t_k$ as the value of ${\nu }_k$, without additional parameters.

The principal results obtained in this paper are presented as follows.

(1)

We examine the application of NL in determining the value t in the Dai–Liao CG method (5).

(2)

A theoretical analysis is conducted to confirm the global convergence of the proposed method.

(3)

A numerical comparison between the proposed FDL algorithm and other known DL algorithms is provided.

3. Fuzzy Neutrosophic Dai–Liao Conjugate Gradient Method

The fuzzy neutrosophic Dai–Liao CG method is initially conceived as a modification of the Dai–Liao CG method (3), where the search directions $\left\{{\mathbf{d}}_k\right\}$ are calculated by the recurrence rule

$${\mathbf{d}}_k=\left\{\begin{array}{cc}-{\mathbf{g}}_0,\hfill & k=0,\hfill \\ -{\mathbf{g}}_k+{\beta }_k^{\mathrm{FDL}}{\mathbf{d}}_{k-1},\hfill & k\ge 1,\hfill \end{array}\right.$$

(19)

where the CG coefficient ${\beta }_k^{\mathrm{FDL}}$ is defined by

$${\beta }_k^{\mathrm{FDL}}=\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}-{\nu }_k\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}},$$

(20)

such that ${\nu }_k$ is a proper fuzzy neutrosophic parameter. Our intention is to define ${\nu }_k$ as a function ${\nu }_k:={\nu }_k\left({\Delta }_k\right)$ of ${\Delta }_k:=f\left({\mathbf{x}}_k\right)-f\left({\mathbf{x}}_{k+1}\right)$. More precisely, ${\nu }_k\left({\Delta }_k\right)$ is defined subject to the following constraint

$$0\le {\nu }_k\left({\Delta }_k\right)\le 1.$$

(21)

It is know that ${\nu }_k\left({\Delta }_k\right)=0$ reduces (2) into

$${\mathbf{d}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}=0.$$

(22)

Hence, Equation (22) can be considered as a reflection of the conjugacy condition, which, in conjunction with (4), determines the HS parameter [37]

$${\beta }_k^{\mathrm{HS}}=\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}.$$

(23)

Alternatively, for ${\nu }_k\left({\Delta }_k\right)=1$, Equation (2) is considered a conjugacy condition that implicitly satisfies the quasi-Newton characteristics. For more details on these cases, see [1,12].

The idea for defining a new parameter $t_k$ in the Dai–Liao CG method (5) comes from the neutrosophic logic. According to this decision, we intend to define $t_k:={\nu }_k\left({\Delta }_k\right)$ with values inside the interval $[0,1]$ according to neutrosophic principles.

The generic layout of the fuzzy neutrosophic Dai–Liao CG method is given in the diagram in Figure 1.

The input of the NLC presented in Figure 1 is ${\Delta }_k:=f\left({\mathbf{x}}_k\right)-f\left({\mathbf{x}}_{k+1}\right)$ and the output is the desired step size ${\nu }_k$. This means that our basic idea is to define ${\nu }_k$ based on two consecutive values of the objective function f. On the other hand, the backtracking line search is responsible for appropriate step lengths ${\alpha }_k$ in (3), and then the descent direction ${\mathbf{d}}_k$ is defined by (19). Using ${\nu }_k$, it is possible to compute ${\beta }_k^{\mathrm{FDL}}$ in (20). Finally, (3) generates a new iterative point $x_{k+1}$.

To develop the FDL method, it is necessary to plan three global steps: neutrosophication, neutrosophic inference engine, and de-neutrosophication (score function).

(1)

Neutrosophication maps the input ${\Delta }_k:=f\left({\mathbf{x}}_k\right)-f\left({\mathbf{x}}_{k+1}\right)$ into neutrosophic ordered triplets $(\mathrm{T}\left({\Delta }_k\right),\mathrm{I}\left({\Delta }_k\right),\mathrm{F}\left({\Delta }_k\right))$. The MFs are defined to improve the CG iterative rule by exploiting numerical experience. The sigmoid function with the slope defined by ${\varsigma }_1$ at the crossover point $\Delta ={\varsigma }_2$ is a proper choice for $\mathrm{T}$:

$$\mathrm{T}(\Delta )=1/\left(1+{\mathrm{e}}^{-{\varsigma }_1(\Delta -{\varsigma }_2)}\right).$$

(24)

A proper choice for $\mathrm{F}$ is the following sigmoid function:

$$\mathrm{F}(\Delta )=1/(1+{\mathrm{e}}^{{\varsigma }_1(\Delta -{\varsigma }_2)}).$$

(25)

The Gaussian function with the standard deviation ${\varsigma }_1$ and the mean ${\varsigma }_2$ defines the indeterminacy $\mathrm{I}$:

$$\mathrm{I}(\Delta )={\mathrm{e}}^{-\frac{{(\Delta -{\varsigma }_2)}^2}{2{\varsigma }_1^2}}.$$

(26)

Then, the neutrosophication of $\Delta \in \mathbb{R}$ is defined as the transition $\Delta \to \langle \mathrm{T}(\Delta ),\mathrm{I}(\Delta ),\mathrm{F}(\Delta )\rangle$, where the MFs are determined in (24)–(26).

(2)

Neutrosophic inference between an input fuzzy set $\mathfrak{I}$ and an output fuzzy set is based on the subsequent “IF–THEN” regulations:

$$\begin{gathered} {\Re }_1:\mathrm{If}\mathfrak{I}=S_P⟹\mathfrak{O}=\{\mathrm{T},\mathrm{I},\mathrm{F}\} \\ {\Re }_2:\mathrm{If}\mathfrak{I}=S_N⟹\mathfrak{O}=\{\mathrm{T},\mathrm{I},\mathrm{F}\}. \end{gathered}$$

Fuzzy sets $S_P$ and $S_N$ point, respectively, to positive or negative errors. Applying the unification $\Re ={\Re }_1\cup {\Re }_2$, we define ${\mathfrak{O}}_i=\mathfrak{I}\circ {\Re }_i$, $i=1,2$, where ∘ denotes the fuzzy transformation. In addition, for a fuzzy triple $\zeta =\{\mathrm{T}\left({\Delta }_k\right),\mathrm{I}\left({\Delta }_k\right),\mathrm{F}\left({\Delta }_k\right)\}$, it is required ${\kappa }_{\mathfrak{I}\circ \Re }(\zeta )={\kappa }_{\mathfrak{I}\circ {\Re }_1}\bigvee {\kappa }_{\mathfrak{I}\circ {\Re }_2}=sup({\kappa }_{\mathfrak{I}}\bigwedge {\kappa }_{{\mathfrak{O}}_i})$, $i=1,2$, where ⋀ and ⋁ denote the (min, max, max) and (max, min, min) operator, respectively. In this research, the centroid defuzzification method is utilized to generate a vector of crisp outputs ${\zeta }^{*}\in {\mathbb{R}}^3$ as follows:

$${\zeta }^{*}=\frac{{\int }_{\mathfrak{O}}\zeta {\kappa }_{\mathfrak{I}\circ \Re }(\zeta )\mathrm{d}\zeta }{{\int }_{\mathfrak{O}}{\kappa }_{\mathfrak{I}\circ \Re }(\zeta )\mathrm{d}\zeta }.$$

(3)

De-neutrosophication is based on the transformation $\langle \mathrm{T}\left({\Delta }_k\right),\mathrm{I}\left({\Delta }_k\right),\mathrm{F}\left({\Delta }_k\right)\rangle \to {\nu }_k\in \mathbb{R}$, resulting in a crisp value ${\nu }_k$, proposed as

$${\nu }_k=2-\left(\mathrm{T}\left({\Delta }_k\right)+\mathrm{I}\left({\Delta }_k\right)+\mathrm{F}\left({\Delta }_k\right)\right).$$

(27)

The diagram in Figure 2 presents the NLC based on the neutrosophic control.

The settings in the NLC employed in numerical testing are arranged in

Table 1. Recommended parameters in NLC.

Set	Membership Function	${\mathit{\varsigma }}_1$	${\mathit{\varsigma }}_2$	Weight
Input	Sigmoid function (24)	1	3	1
	Sigmoid function (25)	1	3	1
	Gaussian function (26)	120	0	1
Output	Score function (27)	-	-	1

.

Our imperative requirement is $0\le {\nu }_k\left({\Delta }_k\right)\le 1$, requested in (21). The fulfillment of those conditions is verified in Lemma 1.

Lemma 1. 

The inequality (21) holds for the given choice of the score function (27) and the parameters given in Table 1.

Proof. 

To prove (21), we need to replace the MFs (24)–(26) in (27). After applying the parameters from Table 1, we obtain

$$\mathrm{T}\left({\Delta }_k\right)=1/\left(1+{\mathrm{e}}^{-(\Delta -3)}\right),\mathrm{F}\left({\Delta }_k\right)=1/\left(1+{\mathrm{e}}^{\Delta -3}\right),\mathrm{I}\left({\Delta }_k\right)={\mathrm{e}}^{-\frac{{\Delta }_k^2}{2\ast {120}^2}}.$$

Elementary calculation provides

$${\nu }_k\left({\Delta }_k\right)=2-\left(\mathrm{T}\left({\Delta }_k\right)+\mathrm{I}\left({\Delta }_k\right)+\mathrm{F}\left({\Delta }_k\right)\right)=1-{\mathrm{e}}^{-\frac{{\Delta }_k^2}{28800}}.$$

(28)

A careful analysis of the function (28) inside the interval ${\Delta }_k\in (-\infty ,+\infty )$ discovers $min{\nu }_k\left({\Delta }_k\right)=0$ and $max{\nu }_k\left({\Delta }_k\right)=1$, which proves the inequality (21).    □

Graphs of $\mathrm{T}\left({\Delta }_k\right),\mathrm{I}\left({\Delta }_k\right),\mathrm{F}\left({\Delta }_k\right)$ are displayed in Figure 3a. The fulfilment of the requirements (21) in the NLC output ${\nu }_k$ generated throughout the described de-neutrosophication is illustrated in Figure 3b.

Remark 1. 

The objective function decreases with the flow of iterations and tends to the minimal value, which means ${lim}_{k\to \infty }{\Delta }_k=0$, i.e., ${lim}_{k\to \infty }{\nu }_k\left({\Delta }_k\right)=0$. Such behavior leads to ${\nu }_k\to 0$ as the minimum of f approaches, so the impact of the proposed neutrosophic strategy decreases and disappears, which agrees with our goal.

Remark 2. 

Obviously, larger values of ${\Delta }_k$ lead to increasing values ${\nu }_k\left({\Delta }_k\right)$ approaching 1, which will be denoted as ${\nu }_k\left({\Delta }_k\right)\nearrow 1$. In addition, based on the limit ${\Delta }_k\to 0$, we anticipate smaller values ${\nu }_k\left({\Delta }_k\right)$ approaching 0, i.e., ${\nu }_k\left({\Delta }_k\right)\searrow 0$, in final iterations. As a result, ${\nu }_k\left({\Delta }_k\right)$ is suitable as an adjustable regulator for the quantity t in the Dai–Liao CG method.

The backtracking line search used in [38] begins at $\alpha =1$ and generates further step sizes, which ensure a decrease in the goal function in each iteration. Algorithm 1, restated from [39], is used to define the principal step size ${\alpha }_k$ in (3).

Algorithm 1 The backtracking line search.

Require:  Objective function $f\left(\mathbf{x}\right)$, foregoing point ${\mathbf{x}}_k$, the search direction ${\mathbf{d}}_k$, a real positive constants $0<\phi <1$, and $0<\omega <0.5$.   1: $\ell =1$.   2: While $f({\mathbf{x}}_k+\ell {\mathbf{d}}_k)>f\left({\mathbf{x}}_k\right)+\omega \ell {\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_k$, do $\ell :=\ell \phi$.   3: Output: ${\alpha }_k=\ell$.

Algorithm 2 of the FDL method is described as follows:

Algorithm 2 Fuzzy neutrosophic Dai–Liao (FDL) conjugate gradient method.

Require:  A starting point ${\mathbf{x}}_0$ and $0<ϵ,\delta <1$.   1: Assign $k=0$ and ${\mathbf{d}}_0=-{\mathbf{g}}_0$.   2: If $$\parallel {\mathbf{g}}_k\parallel \le ϵ\mathrm{and}\frac{|f\left({\mathbf{x}}_{k+1}\right)-f\left({\mathbf{x}}_k\right)|}{1+\left|f\right({\mathbf{x}}_k\left)\right|}\le \delta ,$$ STOP; else go to Step 3.   3: (Backtracking line search) Regulate ${\alpha }_k\in (0,1]$ utilizing Algorithm 1.   4: Calculate ${\mathbf{x}}_{k+1}={\mathbf{x}}_k+{\alpha }_k{\mathbf{d}}_k$.   5: Calculate ${\mathbf{g}}_{k+1}$, ${\mathbf{y}}_k={\mathbf{g}}_{k+1}-{\mathbf{g}}_k$, ${\mathbf{s}}_k={\mathbf{x}}_{k+1}-{\mathbf{x}}_k$.   6: Calculate ${\Delta }_k:=f_k-f_{k+1}$.   7: Calculate $\mathrm{T}\left({\Delta }_k\right),\mathrm{I}\left({\Delta }_k\right)$, and $\mathrm{F}\left({\Delta }_k\right)$ as in (24)–(26).   8: Calculate ${\nu }_k:={\nu }_k\left({\Delta }_k\right)$ using (27).   9: Calculate ${\beta }_{k+1}^{\mathrm{FDL}}$ by (20).   10: Generate ${\mathbf{d}}_{k+1}=-{\mathbf{g}}_{k+1}+{\beta }_{k+1}^{\mathrm{FDL}}{\mathbf{d}}_k$.   11: Set $k:=k+1,$ and go to Step 2.

4. Convergence Examination

The subsequent assumptions are necessary during the theoretical examination of FDL iterations.

Assumption 1. 

$\left(1\right)$ The level set $\mho =\{\mathbf{x}\in {\mathbb{R}}^n|f\left(\mathbf{x}\right)\le f\left({\mathbf{x}}_0\right)\}$ of the iterative process (3) is bounded.

$\left(2\right)$ The objective f is continuously differentiable in a neighborhood $\mathcal{P}$ of ℧ with the Lipschitz continuous gradient $\mathbf{g}$. Such supposition initiates the existence of a constant $L>0$ which fulfills the inequality

$$\parallel \mathbf{g}\left(\mathbf{u}\right)-\mathbf{g}\left(\mathbf{v}\right)\parallel \le L\parallel \mathbf{u}-\mathbf{v}\parallel ,\forall \mathbf{u},\mathbf{v}\in \mathcal{P}.$$

(29)

Assumption 1 provides the existence of quantities ${\rm Y}$ and $\gamma$ that fulfil

$$\parallel \mathbf{u}-\mathbf{v}\parallel \le {\rm Y},\forall \mathbf{u},\mathbf{v}\in \mathcal{P}$$

(30)

and

$$\parallel \mathbf{g}\left(\mathbf{u}\right)\parallel \le \gamma ,\forall \mathbf{u}\in \mathcal{P}.$$

(31)

If Assumption 1 holds, given the uniform convexity of f, there exists $\theta >0$, satisfying

$${(\mathbf{g}\left(\mathbf{u}\right)-\mathbf{g}\left(\mathbf{v}\right))}^{\mathrm{T}}(\mathbf{u}-\mathbf{v})\ge \theta {\parallel \mathbf{u}-\mathbf{v}\parallel }^2,\forall \mathbf{u},\mathbf{v}\in \mho ,$$

(32)

or, equivalently,

$$f\left(\mathbf{u}\right)\ge f\left(\mathbf{v}\right)+g{\left(\mathbf{v}\right)}^{\mathrm{T}}(\mathbf{u}-\mathbf{v})+\frac{\theta }{2}{\parallel \mathbf{u}-\mathbf{v}\parallel }^2,\forall \mathbf{u},\mathbf{v}\in \mho .$$

(33)

From (32) and (33), it follows

$${\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\ge \theta {\parallel {\mathbf{s}}_{k-1}\parallel }^2$$

(34)

and

$$f\left({\mathbf{x}}_{k-1}\right)-f\left({\mathbf{x}}_k\right)\ge -\mathbf{g}{\left({\mathbf{x}}_k\right)}^{\mathrm{T}}{\mathbf{s}}_{k-1}+\frac{\theta }{2}{\parallel {\mathbf{s}}_{k-1}\parallel }^2.$$

(35)

By (29) and (34), one concludes

$$\theta \parallel {\mathbf{s}}_{k-1}{\parallel }^2\le {\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\le L{\parallel {\mathbf{s}}_{k-1}\parallel }^2,$$

(36)

and, further, $\theta \le L$.

The inequality (36) implies

$${\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}={\alpha }_{k-1}{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}>0.$$

(37)

Taking into account ${\alpha }_{k-1}>0$ and the last inequality, we conclude

$${\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}>0.$$

(38)

Lemma 2 

([40,41]). Let the constraints in Assumption 1 hold and the points $\left\{{\mathbf{x}}_k\right\}$ be produced by the iterations (3) and (4). Then, the following inequality is satisfied:

$$\sum _{k=0}^{\infty }{\displaystyle \frac{\parallel {\mathbf{g}}_k{\parallel }^4}{\parallel {\mathbf{d}}_k{\parallel }^2}}<+\infty .$$

(39)

Lemma 3. 

Observe the suggested fuzzy neutrosophic Dai–Liao CG method defined by (3), (19), (20). If the search procedure enables (38) for all $k\ge 0$, it follows

$${\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_k\le -c{\parallel {\mathbf{g}}_k\parallel }^2$$

(40)

for some, $c\ge 0$.

Proof. 

In the initial stage, it follows ${\mathbf{g}}_0^{\mathrm{T}}{\mathbf{d}}_0=-{\parallel {\mathbf{g}}_0\parallel }^2$. Using $c=1$, it can be concluded that (40) is satisfied in the initial stage $k=0$. Assume (40) for some $k\ge 1$. Applying the inner product between the left- and right-hand sides of (19) and ${\mathbf{g}}_k^{\mathrm{T}}$, it is concluded

$$\begin{array}{ll}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_k& =-{\parallel {\mathbf{g}}_k\parallel }^2+{\beta }_k^{\mathrm{FDL}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\\ & =-{\parallel {\mathbf{g}}_k\parallel }^2+\left(\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}-{\nu }_k\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\right){\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\\ & =-{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}-{\nu }_k\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\\ & =-{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}-{\nu }_k\frac{{\alpha }_{k-1}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\\ & =-{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}-{\nu }_k\frac{{\alpha }_{k-1}{\left({\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right)}^2}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}.\end{array}$$

(41)

Using (21) in conjunction with (38) and ${\alpha }_{k-1}>0$, we conclude

$${\nu }_k\frac{{\alpha }_{k-1}{\left({\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right)}^2}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\ge 0.$$

(42)

Now, from (41) and (42), it follows

$$\begin{array}{ll}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_k& \le -{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\\ & =\frac{-\parallel {\mathbf{g}}_k{\parallel }^2{\left({\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2+\left({\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)\left({\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right)\left({\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}{{\left({\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2}.\end{array}$$

(43)

Applying the inequality $P^{\mathrm{T}}Q\le \frac{1}{2}({\parallel P\parallel }^2+{\parallel Q\parallel }^2)$ to the equality (43) with $P=\frac{1}{\sqrt{2}}\left({\mathbf{d}}_{k-1}^T{\mathbf{y}}_{k-1}\right){\mathbf{g}}_k$ and $Q=\sqrt{2}\left({\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right){\mathbf{y}}_{k-1}$, it is obtained

$$\begin{array}{ll}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_k& \le \frac{-{\parallel {\mathbf{g}}_k\parallel }^2{\left({\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2+\frac{1}{2}\left({\parallel \frac{1}{\sqrt{2}}\left({\mathbf{d}}_{k-1}^T{\mathbf{y}}_{k-1}\right){\mathbf{g}}_k\parallel }^2+{\parallel \sqrt{2}\left({\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right){\mathbf{y}}_{k-1}\parallel }^2\right)}{{\left({\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2}\\ & =\frac{-{\parallel {\mathbf{g}}_k\parallel }^2{\left({\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2+\frac{1}{2}\left(\frac{1}{2}{\left({\mathbf{d}}_{k-1}^T{\mathbf{y}}_{k-1}\right)}^2{\parallel {\mathbf{g}}_k\parallel }^2+2{\left({\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right)}^2{\parallel {\mathbf{y}}_{k-1}\parallel }^2\right)}{{\left({\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2}\\ & =\frac{-{\parallel {\mathbf{g}}_k\parallel }^2{\left({\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2+\frac{1}{4}{\left({\mathbf{d}}_{k-1}^T{\mathbf{y}}_{k-1}\right)}^2{\parallel {\mathbf{g}}_k\parallel }^2+{\left({\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right)}^2{\parallel {\mathbf{y}}_{k-1}\parallel }^2}{{\left({\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2}\\ & =-{\parallel {\mathbf{g}}_k\parallel }^2+\frac{1}{4}{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{\left({\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right)}^2{\parallel {\mathbf{y}}_{k-1}\parallel }^2}{{\left({\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2}\\ & \le -{\parallel {\mathbf{g}}_k\parallel }^2+\frac{1}{4}{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{\alpha }_{k-1}^2\left(\right|{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}{\left|\right)}^2{\parallel {\mathbf{y}}_{k-1}\parallel }^2}{{\left({\alpha }_{k-1}{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2}\\ & \le -{\parallel {\mathbf{g}}_k\parallel }^2+\frac{1}{4}{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{\alpha }_{k-1}^2{\parallel {\mathbf{g}}_k\parallel }^2{\parallel {\mathbf{d}}_{k-1}\parallel }^2{\parallel {\mathbf{y}}_{k-1}\parallel }^2}{{\left({\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right)}^2}\\ & \le -{\parallel {\mathbf{g}}_k\parallel }^2+\frac{1}{4}{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{\parallel {\mathbf{g}}_k\parallel }^2{\parallel {\alpha }_{k-1}{\mathbf{d}}_{k-1}\parallel }^2{\parallel {\mathbf{y}}_{k-1}\parallel }^2}{{\theta }^2{\parallel {\mathbf{s}}_{k-1}\parallel }^4}\\ & \le -{\parallel {\mathbf{g}}_k\parallel }^2+\frac{1}{4}{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{\parallel {\mathbf{g}}_k\parallel }^2{\parallel {\mathbf{s}}_{k-1}\parallel }^2{\parallel {\mathbf{y}}_{k-1}\parallel }^2}{{\theta }^2{\parallel {\mathbf{s}}_{k-1}\parallel }^4}\\ & \le -{\parallel {\mathbf{g}}_k\parallel }^2+\frac{1}{4}{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{\parallel {\mathbf{g}}_k\parallel }^2{\parallel {\mathbf{y}}_{k-1}\parallel }^2}{{\theta }^2{\parallel {\mathbf{s}}_{k-1}\parallel }^2}\\ & \le -{\parallel {\mathbf{g}}_k\parallel }^2+\frac{1}{4}{\parallel {\mathbf{g}}_k\parallel }^2+\frac{L^2{\parallel {\mathbf{g}}_k\parallel }^2{\parallel {\mathbf{s}}_{k-1}\parallel }^2}{{\theta }^2{\parallel {\mathbf{s}}_{k-1}\parallel }^2}\\ & =-{\parallel {\mathbf{g}}_k\parallel }^2+\frac{1}{4}{\parallel {\mathbf{g}}_k\parallel }^2+\frac{L^2}{{\theta }^2}{\parallel {\mathbf{g}}_k\parallel }^2\\ & =-\left(1-\frac{1}{4}-\frac{L^2}{{\theta }^2}\right){\parallel {\mathbf{g}}_k\parallel }^2=-\left(\frac{3}{4}-\frac{L^2}{{\theta }^2}\right){\parallel {\mathbf{g}}_k\parallel }^2.\end{array}$$

(44)

The requirement (40) is satisfied for $c=\left(\frac{3}{4}-\frac{L^2}{{\theta }^2}\right)$ in (44) and an arbitrary $k\ge 0$. □

Theorem 1 confirms the global convergence of the FDL flow.

Theorem 1. 

Let the constraints in Assumption 1 be valid and f be uniformly convex. Under these conditions, iterations $\left\{{\mathbf{x}}_k\right\}$ generated by (3), (19), and (20) fulfill the limit relation

$$\underset{k\to \infty }{lim\; inf}\parallel {\mathbf{g}}_k\parallel =0.$$

(45)

Proof. 

Suppose the opposite. Since (45) is not valid, we conclude the existence of $c_1>0$, satisfying

$$\parallel {\mathbf{g}}_k\parallel \ge c_1,\mathrm{for}\mathrm{all}k.$$

(46)

Squaring both sides of (19), one derives

$${\parallel {\mathbf{d}}_k\parallel }^2={\parallel {\mathbf{g}}_k\parallel }^2-2{\beta }_k^{\mathrm{FDL}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}+{\left({\beta }_k^{\mathrm{FDL}}\right)}^2{\parallel {\mathbf{d}}_{k-1}\parallel }^2.$$

(47)

Taking into account (20), we obtain

$$\begin{array}{ll}-2{\beta }_k^{\mathrm{FDL}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}& =-2\left(\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}-{\nu }_k\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\right){\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\\ & =-2\left(\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}-{\nu }_k\frac{{\alpha }_{k-1}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right)\\ & =-2\left(\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}-{\nu }_k\frac{{\alpha }_{k-1}{\left({\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right)}^2}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\right).\end{array}$$

(48)

Now, from (42), it follows

$$\begin{array}{ll}-2{\beta }_k^{\mathrm{EDL}}{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}& \le 2\left|\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\right|\left|{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{d}}_{k-1}\right|\le 2\frac{\parallel {\mathbf{g}}_k\parallel \parallel {\mathbf{y}}_{k-1}\parallel }{\left|{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}\right|}\parallel {\mathbf{g}}_k\parallel \parallel {\mathbf{d}}_{k-1}\parallel \\ & =2\frac{{\alpha }_{k-1}{\parallel {\mathbf{g}}_k\parallel }^2\parallel {\mathbf{y}}_{k-1}\parallel \parallel {\mathbf{d}}_{k-1}\parallel }{{\alpha }_{k-1}{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\\ & =2\frac{{\parallel {\mathbf{g}}_k\parallel }^2\parallel {\mathbf{y}}_{k-1}\parallel \parallel {\mathbf{s}}_{k-1}\parallel }{{\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\\ & \le 2\frac{{\parallel {\mathbf{g}}_k\parallel }^{2}L\parallel {\mathbf{s}}_{k-1}\parallel \parallel {\mathbf{s}}_{k-1}\parallel }{\theta {\parallel {\mathbf{s}}_{k-1}\parallel }^2}\\ & =\frac{2L}{\theta }{\parallel {\mathbf{g}}_k\parallel }^2.\end{array}$$

(49)

Now, an application of (20) initiates

$$\begin{array}{ll}{\beta }_k^{\mathrm{FDL}}& =\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}-{\nu }_k\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}=\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}-{\nu }_k{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\\ & \le \left|\frac{{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}-{\nu }_k{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}}{{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\right|={\alpha }_{k-1}\frac{\left|{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}-{\nu }_k{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}\right|}{{\alpha }_{k-1}{\mathbf{d}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\\ & ={\alpha }_{k-1}\frac{\left|{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}-{\nu }_k{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}\right|}{{\mathbf{s}}_{k-1}^{\mathrm{T}}{\mathbf{y}}_{k-1}}\\ & \le {\alpha }_{k-1}\frac{\left|{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{y}}_{k-1}-{\nu }_k{\mathbf{g}}_k^{\mathrm{T}}{\mathbf{s}}_{k-1}\right|}{\theta {\parallel {\mathbf{s}}_{k-1}\parallel }^2}={\alpha }_{k-1}\frac{\left|{\mathbf{g}}_k^{\mathrm{T}}({\mathbf{y}}_{k-1}-{\nu }_k{\mathbf{s}}_{k-1})\right|}{\theta {\alpha }_{k-1}^2{\parallel {\mathbf{d}}_{k-1}\parallel }^2}\\ & \le \frac{\parallel {\mathbf{g}}_k\parallel \left(\parallel {\mathbf{y}}_{k-1}\parallel +{\nu }_k\parallel {\mathbf{s}}_{k-1}\parallel \right)}{\theta {\alpha }_{k-1}{\parallel {\mathbf{d}}_{k-1}\parallel }^2}\\ & \le \frac{\parallel {\mathbf{g}}_k\parallel \left(L\parallel {\mathbf{s}}_{k-1}\parallel +{\nu }_k)\parallel {\mathbf{s}}_{k-1}\parallel \right)}{\theta {\alpha }_{k-1}{\parallel {\mathbf{d}}_{k-1}\parallel }^2}=\frac{\parallel {\mathbf{g}}_k\parallel \left(L+{\nu }_k\right)\parallel {\mathbf{s}}_{k-1}\parallel }{\theta {\alpha }_{k-1}{\parallel {\mathbf{d}}_{k-1}\parallel }^2}\\ & =\frac{\parallel {\mathbf{g}}_k\parallel \left(L+{\nu }_k\right){\alpha }_{k-1}\parallel {\mathbf{d}}_{k-1}\parallel }{\theta {\alpha }_{k-1}{\parallel {\mathbf{d}}_{k-1}\parallel }^2}\\ & =\frac{\left(L+{\nu }_k\right)\parallel {\mathbf{g}}_k\parallel }{\theta \parallel {\mathbf{d}}_{k-1}\parallel }.\end{array}$$

(50)

Further, (21) and (50) lead to

$${\beta }_k^{\mathrm{FDL}}\le \frac{\left(L+1\right)\parallel {\mathbf{g}}_k\parallel }{\theta \parallel {\mathbf{d}}_{k-1}\parallel }.$$

(51)

Using (49) and (51) in (47), we obtain

$$\begin{array}{ll}\parallel {\mathbf{d}}_k{\parallel }^2& \le {\parallel {\mathbf{g}}_k\parallel }^2+\frac{2L}{\theta }{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{(L+1)}^2{\parallel {\mathbf{g}}_k\parallel }^2}{{\theta }^2{\parallel {\mathbf{d}}_{k-1}\parallel }^2}{\parallel {\mathbf{d}}_{k-1}\parallel }^2\\ & ={\parallel {\mathbf{g}}_k\parallel }^2+\frac{2L}{\theta }{\parallel {\mathbf{g}}_k\parallel }^2+\frac{{(L+1)}^2{\parallel {\mathbf{g}}_k\parallel }^2}{{\theta }^2}\\ & =\left(1+\frac{2L}{\theta }+\frac{{(L+1)}^2}{{\theta }^2}\right){\parallel {\mathbf{g}}_k\parallel }^2\\ & =\left(\frac{\theta +2L}{\theta }+\frac{{(L+1)}^2}{{\theta }^2}\right){\parallel {\mathbf{g}}_k\parallel }^2\\ & =\frac{(\theta +2L)\theta +{(L+1)}^2}{{\theta }^2}{\parallel {\mathbf{g}}_k\parallel }^2.\end{array}$$

(52)

Dividing both sides of (52) by ${∥{\mathbf{g}}_k∥}^4$ and utilizing (46), it can be decided

$$\begin{gathered} \frac{\parallel {\mathbf{d}}_k{\parallel }^2}{\parallel {\mathbf{g}}_k{\parallel }^4}\le \frac{(\theta +2L)\theta +{(L+1)}^2}{{\theta }^2}·\frac{1}{c_1^2}, \\ \frac{\parallel {\mathbf{g}}_k{\parallel }^4}{\parallel {\mathbf{d}}_k{\parallel }^2}\ge \frac{{\theta }^2·c_1^2}{(\theta +2L)\theta +{(L+1)}^2}. \end{gathered}$$

(53)

The inequalities in (53) imply

$$\sum _{k=0}^{\infty }\frac{\parallel {\mathbf{g}}_k{\parallel }^4}{\parallel {\mathbf{d}}_k{\parallel }^2}\ge \sum _{k=0}^{\infty }\frac{{\theta }^2·c_1^2}{(\theta +2L)\theta +{(L+1)}^2}=\infty .$$

(54)

Therefore, $\parallel {\mathbf{g}}_k\parallel \ge c_1$ causes a contradiction with Lemma 2. □

5. Numerical Results

In current section, the numerical results obtained by the FDL method are analyzed and compared with the numerical results generated by the EDL [21] and DL [1] methods.

All the algorithms were implemented in Matlab R2017a and executed on a 64-bit Lenovo laptop (Intel Core i3 $2.0$ GHz, RAM 8 GB) with the Windows 10 operating system. The implementation of the FDL method is based on Algorithm 2, while the implementation of EDL and DL is based on algorithms given in [21] and [1], respectively.

The numerical testing is performed on 50 test functions collected in [42,43], with dimensions in the range $[100,20000]$. All three tested methods start from the same initial point ${\mathbf{x}}_0$ for each test example. Each case is evaluated 10 times with progressively increased dimensions $n={10}^2$, $5\times {10}^2,{10}^3,3\times {10}^3,\times {10}^3,7\times {10}^3,8\times {10}^3$, ${10}^4$, $1.5\times {10}^4$, and $2\times {10}^4$.

The uniform terminating criteria for the observed DL, EDL, and FDL algorithms are

$$\parallel {\mathbf{g}}_k\parallel \le ϵ\mathrm{and}\frac{|f\left({\mathbf{x}}_{k+1}\right)-f\left({\mathbf{x}}_k\right)|}{1+\left|f\right({\mathbf{x}}_k\left)\right|}\le \delta ,ϵ={10}^{-6},\delta ={10}^{-16}.$$

We will evaluate the efficiency of the FDL method and compare it with the EDL and DL methods under the backtracking search that uses the values $\omega =0.0001$ and $\phi =0.8$.

Summary numerical results for DL, FDL, and EDL methods, performed on 50 test functions, are shown in

Table 2. Summary of numerical results on unconstrained problems of DL, FDL, and EDL methods for the Nitr, Nfe, and Tcpu.

Test Function	DL	FDL	EDL
	Nitr/Nfe/Tcpu	Nitr/Nfe/Tcpu	Nitr/Nfe/Tcpu
Extended Penalty	1905/77,578 /32.438	1610/62,534/24.266	2304/82,602/39.344
Perturbed Quadratic	14,555/60,6750/379.359	10,800/440213/206.5	10,012/408,474/248.5
Raydan 1	4337/114,595/98.984	5497/122,843/76.813	4194/109,164/96.938
Raydan 2	1427/2864/3.188	67/144/0.281	2,572,540/5,145,090/894.453
Diagonal 1	5809/223,750/245.578	5488/212,491/227.781	4673/178,295/219.109
Diagonal 3	5247/196,745/423.766	4531/168,162/307.594	4596/171636/366.203
Hager	1742/31,516/103.672	1242/22,799/47.063	1940/33,206/98.766
Generalized Tridiagonal 1	2058/32,313/49.5	2160/32,033/27.5	2161/33285/44.703
Extended Tridiagonal 1	310/2932/8.391	182/2501/6.297	308/4129/12.766
Extended TET	1140/9840/11.031	619/5808/5.484	749/6362/5.969
Diagonal 5	1394/2798/6.938	60/130/0.609	3,053,907/6,107,824/3124.875
Extended Himmelblau	50/2431/1.016	51/2602/0.813	50/2413/0.938
Perturbed quadratic diagonal	1837/69,156/18.453	1261/36,785/13.875	2157/86,977/34.797
Quadratic QF1	13,895/526,995/187.313	21,989/846,402/376.156	10,199/379,554/122.844
Extended quadratic penalty QP1	1080/17,440/9.922	1524/23,840/8.25	1157/18043/9.109
Extended quadratic penalty QP2	218/9479/11.047	112/5513/4.953	218/9194/8.906
Quadratic QF2	19,211/847,031/348.781	18,861/816,310/225.891	15,555/689,736/250.891
Extended quadratic exponential EP1	1254/3443/3.172	56/404/0.516	21,431/43,829/7.531
Extended Tridiagonal 2	22,468/998,473/549.484	3668/114,169/87.438	10989/510713/93.609
TRIDIA (CUTE)	33,278/1,647,913/967.234	40,156/1,977,068/950.547	29,133/1,428,866/675.422
ARWHEAD (CUTE)	1624/81,625/44.875	1529/72,379/31.594	1219/57140/28.672
Almost Perturbed Quadratic	14,904/621,925/259.797	19,675/829,784/357.359	13,201/543,372/188.047
LIARWHD (CUTE)	30/2705/1.281	30/2732/1.25	30/2739/1.438
POWER (CUTE)	532,442/44,419,504/16,742.672	580790/48609979/17435.609	629342/52,431,424/23,630.781
ENGVAL1 (CUTE)	2489/33,103/13.781	2400/32,299/10.719	1975/27,260/12.922
INDEF (CUTE)	21/1924/2.125	26/2238/2.5	30/2610/4.266
Diagonal 6	1583/3197/4.531	74/185/0.359	7,052,401/14105032/5037.219
DIXON3DQ (CUTE)	320,921/1775846/1083.281	229,757/1,368,033/727.172	257,451/1,517,252/1045.328
COSINE (CUTE)	20/1600/1.891	20/1697/1.891	20/1700/2
BIGGSB1 (CUTE)	249,919/1,400,798/832.375	259,475/1,549,293/810.766	236,612/1,389,720/945.672
Generalized Quartic	866/11,273/3.984	1099/8951/4.063	959/10,662/3.125
Diagonal 7	1453/4564/6.875	68/162/0.469	469,477/940,686/140.172
Diagonal 8	1371/3962/5.359	67/199/0.422	594,522/1,193,760/195.094
Full Hessian FH3	2237/6202/7.125	52/513/0.688	767,988/1,537,759/188.469
Diagonal 9	3312/138,545/225.719	5344/217,150/224.906	4520/189,307/260.453
HIMMELH (CUTE)	20/1690/4.797	20/1758/4.531	20/1760/4.891
FLETCHCR (CUTE)	303,212/10,189,775/5073.688	300,227/10011849/4704.125	289,670/9,702,961/4411.453
Extended BD1 (Block Diagonal)	1597/16,783/7.625	1227/15,639/5.875	1200/12,605/6.625
Extended Maratos	72/3366/1.188	50/2069/0.719	40/1975/0.75
Extended Cliff	234/2992/2.078	217/6000/4.891	950/13,187/6.188
Extended Hiebert	70/7215/1.938	70/7220/1.828	70/7228/1.859
NONDIA (CUTE)	33/3066/1.375	30/2829/1.266	32/3031/1.625
NONDQUAR (CUTE)	58/4652/18.047	45/3666/17.219	86/4989/19.016
DQDRTIC (CUTE)	3456/87,105/26.453	2327/59,047/16.406	3637/92315/34.953
Extended Freudenstein and Roth	1376/46,597/10.734	3390/111,830/28.516	2018/66,654/16.172
Generalized Rosenbrock	282,948/8,410,218/4125.516	280,440/8,335,396/4088.547	281,792/8,373,946/4055.172
Extended White and Holst	76/5794/9.219	50/3171/7.281	59/4022/11.563
Extended Beale	118/6791/14.047	72/3118/5.906	181/4748/6.75
EG2 (CUTE)	507/29,388/47.547	697/48,512/119.875	811/39,769/122.469
EDENSCH (CUTE)	1694/23,160/89.453	2089/27,821/83.266	1684/22,731/116.844

, where ‘Test function’, ‘Nitr’, ‘Nfe’, and ‘Tcpu’ represent the name of the tested function, total number of iterative steps, total number of function evaluations, and the running time, respectively. The best results between the DL, FDL, and EDL methods in Table 2 are marked by bold text in shaded cells.

To visually compare the performance of the opposed methods, we used the performance profiles technique [44] on the numerical results corresponding to Nitr, Nfe, and Tcpu criteria generated by the DL, FDL, and EDL methods. An upper graph in a performance profile corresponds to the method that shows better performance. The vertical axis of each performance profile in the figures indicates the percentage of test functions for which the considered method is the winner between compared methods, whereby the right-hand side corresponds to the percentage of successfully solved test functions.

Figure 4 and Figure 5 plot the performance profiles for the data in Table 2. The graphs in Figure 4 illustrate the performance profiles Nitr and Nfe for DL, FDL, and EDL iterations based on the data from Table 2. In Figure 4a, it is noticeable that the DL, FDL, and EDL methods can solve all the tested functions. Numerical experience shows that the FDL method produces the best results in $54.0$% (27 out of 50) of test functions compared with DL ($26.0$% (13 out of 50)) and EDL ($38.0$% (19 out of 50)). From Figure 4a, the FDL graph touches the top first, so FDL is the best relative to the other examined methods concerning the Nitr criterion.

Figure 4b indicates that the FDL graph is the most efficient and successfully solves all test cases. In addition, the obtained numerical results confirm that FDL performs well in most cases. Specifically, FDL is the fastest because it solves about $48.0\%$ (24 out of 50) of the tested functions with the least Nfe compared to the DL and EDL methods. Meanwhile, DL and EDL are superior in solving $22.0\%$ (11 out of 50) and $30.0\%$ (15 out of 50) test functions, respectively. Hence, FDL is superior compared to the DL and EDL methods for the given test functions.

Figure 5 shows Tcpu performance profile graphs of the DL, FDL, and EDL methods. It is observable that DL, FDL, and EDL are able to solve all the tested functions. Further examination leads to the conclusion that the FDL method is the best in $74.0$% (thirty-seven out of fifty) test cases compared to DL ($12.0$% (six out of fifty)) and EDL ($16.0$% (eight out of fifty)). Analyzing the graphs in Figure 5, a clear conclusion is that the FDL graph first achieves the top, which certifies its superiority in terms of Tcpu.

The general conclusion from the results presented in Table 2 is that one of the methods DL, FDL, and EDL is the absolute winner in thirty-two tested functions relative to all three criteria (Nitr, Nfe, and Tcpu). Further analysis of these data leads to the following conclusions:

• the DL method is an absolute winner in five cases;
• the FDL method is an absolute winner in twenty-one cases;
• the EDL method is an absolute winner in six cases.

In addition to these facts, the data contained in Table 2 show that the FDL method is better in six cases than the other two methods in terms of Nitr. The FDL method is better in three cases compared to DL and EDL in terms of Nfe. The FDL method achieves the best results in sixteen test cases in terms of Tcpu.

The comparisons of the DL, FDL, and EDL methods are shown in Figure 4 and Figure 5. As can be noticed, FDL is superior compared to the DL and EDL methods with respect to three criteria (iterations, function evaluations, and processor time). Moreover, the numerical results in Table 2 indicate that the FDL method is the absolute winner. Thus, numerical experiments indicate the effectiveness of the proposed FDL method.

6. Conclusions

In the current research, we propose a novel approach to determining the parameter t in Dai–Liao CG iterations. The new approach is based on finding suitable values for a non-negative parameter t in the DL method using neutrosophic logic. An original strategy in defining the Dai–Liao CG parameter ${\beta }_k^{\mathrm{FDL}}$ is suggested utilizing $t:={\nu }_k\left({\Delta }_k\right)$ in (20), and a novel fuzzy neutrosophic Dai–Liao (FDL) CG method is presented.

Numerical experiments and comparisons with some well-known CG methods and the theoretical convergence analysis demonstrate the effectiveness of the proposed method. The numerical testing and initiated comparison are based on standard performance profiles, such as total number of iterations (Nitr), total number of function evaluations (Nfe), and runtime (Tcpu). Analysis of the obtained numerical results revealed that the FDL method is the most efficient.

We are assured that the proposed methods will serve as motivation for further investigation in defining improvements of DL iterations guided by the neutrosophic logic.

Future scientific research in this area can continue in several directions. Previous research has shown the effectiveness of the neutrosophic principle in scaled gradient descent methods and the DL class of CG methods. The challenge is to apply such a principle to other nonlinear optimization methods. On the other hand, there is a wide range of possibilities for defining further neutrosophication and de-neutrosophication rules, which can be considered in future research. Finally, there is a great opportunity to improve the neutrosophic inference engine used in this research.